The Python tempfile Module

Introduction

Temporary files, or "temp files", are mainly used to store intermediate information on disk for an application. These files are normally created for different purposes such as temporary backup or if the application is dealing with a large dataset bigger than the system's memory, etc. Ideally, these files are located in a separate directory, which varies on different operating systems, and the name of these files are unique. The data stored in temporary files is not always required after the application quits, so you may want these files to be deleted after use.

Python provides a module known as tempfile, which makes creating and handling temporary files easier. This module provides a few methods to create temporary files and directories in different ways. tempfile comes in handy whenever you want to use temporary files to store data in a Python program. Let's take a look at a couple of different examples on how the tempfile module can be used.

Creating a Temporary File

Suppose your application needs a temporary file for use within the program, i.e. it will create one file, use it to store some data, and then delete it after use. To achieve this, we can use the TemporaryFile() function.

This function will create one temporary file to the default tempfile location. This location may be different between operating systems. The best part is that the temporary file created by TemporaryFile() will be removed automatically whenever the file is closed. Also, it does not create any reference to this file in the system's file system table. This makes it private to the current application i.e. no other program will be able to open the file.

Let's take a look at the below Python program to see how it works:

import tempfile #1

print("Creating one temporary file...")

temp = tempfile.TemporaryFile() #2

try:
    print("Created file is:", temp) #3
    print("Name of the file is:", temp.name) #4
finally:
    print("Closing the temp file")
    temp.close() #5

It will print the below output:

$ python3 temp-file.py
Creating one temporary file...
Created file is: <_io.BufferedRandom name=4>
Name of the file is: 4
Closing the temp file

1. To create one temporary file in Python, you need to import the tempfile module.
2. As explained above, we have created the temporary file using the TemporaryFile() function.
3. From the output, you can see that the created object is actually not a file, it is a file-like object. And, the mode parameter (not shown in our example) of the created file is w+b, i.e. you can read and write without being closed.
4. The temporary file created has no name.
5. Finally, we are closing the file using the close() method. It will be destroyed after it is closed.

One thing we should point out is that the file created using the TemporaryFile() function may or may not have a visible name in the file system. On Unix, the directory entry for the file is removed automatically after it is created, although this is not supported on other platforms. Normally TemporaryFile() is the ideal way to create one temporary storage area for any program in Python.

Create a Named Temporary File

In our previous example, we have seen that the temporary file created using the TemporaryFile() function is actually a file-like object without an actual file name. Python also provides a different method, NamedTemporaryFile(), to create a file with a visible name in the file system. Other than providing a name to the tempfile, NamedTemporaryFile() works the same as TemporaryFile(). Now let's use the same above example to create a named temporary file:

import tempfile

print("Creating one named temporary file...")

temp = tempfile.NamedTemporaryFile()

try:
    print("Created file is:", temp)
    print("Name of the file is:", temp.name)
finally:
    print("Closing the temp file")
    temp.close()

Running this code will print output similar to the following:

$ python3 named-temp-file.py
Creating one named temporary file...
Created file is: <tempfile._TemporaryFileWrapper object at 0x103f22ba8>
Name of the file is: /var/folders/l7/80bx27yx3hx_0_p1_qtjyyd40000gn/T/tmpa3rq8lon
Closing the temp file

So, the created file actually has a name this time. The advantage of NamedTemporaryFile() is that we can save the name of the created temp files and use them later before closing or destroying it. If the delete parameter is set to False, then we can close the file without it being destroyed, allowing us to reopen it later on.

Providing a Suffix or Prefix to the Name

Sometimes we need to add a prefix or suffix to a temp-file's name. It will help us to identify all temp files created by our program.

To achieve this, we can use the same NamedTemporaryFile function defined above. The only thing we need to add is two extra parameters while calling this function: suffix and prefix

import tempfile

temp = tempfile.NamedTemporaryFile(prefix="dummyPrefix_", suffix="_dummySuffix")

try:
    print("Created file is:", temp)
    print("Name of the file is:", temp.name)
finally:
    temp.close()

Running this code will print the following output:

$ python3 prefix-suffix-temp-file.py
Created file is: <tempfile._TemporaryFileWrapper object at 0x102183470>
Name of the file is: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T/dummyPrefix_uz63brcp_dummySuffix

So, if we pass the two extra arguments suffix and prefix to the NamedTemporaryFile() function, it will automatically add those in the start and end of the file name.

Finding the Default Location of Temp Files

The tempfile.tempdir variable holds the default location for all temporary files. If the value of tempdir is None or unset, Python will search a standard list of directories and set tempdir to the first directory value, but only if the calling program can create a file in it. The following are the list of directories it will scan, in this order:

1. The directory named by the TMPDIR environment variable
2. The directory named by the TEMP environment variable
3. The directory named by the TMP environment variable
4. Platform-specific directories:
   1. On windows, C:\TEMP, C:\TMP, \TEMP, and \TMP, in the same order.
   2. On other platforms, /tmp, /var/tmp, and /usr/tmp, in the same order.
5. The current working directory.

To find out the default location of temporary files, we can call the tempfile.gettempdir() method. It will return the value of tempdir if it is not None. Otherwise it will first search for the directory location using the steps mentioned above and then return the location.

import tempfile

print("Current temp directory:", tempfile.gettempdir())

tempfile.tempdir = "/temp"

print("Temp directory after change:", tempfile.gettempdir())

If you will run the above program, it will print an output similar to the following:

$ python3 dir-loc-temp-file.py
Current temp directory: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T
Temp directory after change: /temp

You can see that the first temp directory location is the system-provided directory location and the second temp directory is the same value as the one that we have defined.

Reading and Writing Data from Temp Files

We have learned how to create a temporary file, create a temporary file with a name, and how to create a temporary file with a suffix and/or prefix. Now, let's try to understand how to actually read and write data from a temporary file in Python.

Reading and writing data from a temporary file in Python is pretty straightforward. For writing, you can use the write() method and for reading, you can use the read() method. For example:

import tempfile

temp = tempfile.TemporaryFile()

try:
    temp.write(b'Hello world!')
    temp.seek(0)

    print(temp.read())
finally:
    temp.close()

This will print the output as b'Hello world!' since the write() method takes input data in bytes (hence the b prefix on the string).

If you want to write text data into a temp file, you can use the writelines() method instead. For using this method, we need to create the tempfile using w+t mode instead of the default w+b mode. To do this, a mode param can be passed to TemporaryFile() to change the mode of the created temp file.

import tempfile

temp = tempfile.TemporaryFile(mode='w+t')

try:
    temp.writelines("Hello world!")
    temp.seek(0)

    print(temp.read())
finally:
    temp.close()

Unlike the previous example, this will print "Hello World" as the output.

Create a Temporary Directory

If your program has several temporary files, it may be more convenient to create one temporary directory and put all of your temp files inside of it. To create a temporary directory, we can use the TemporaryDirectory() function. After all temp files are closed, we need to delete the directory manually.

import tempfile

with tempfile.TemporaryDirectory() as tmpdirname:
    print('Created temporary directory:', tmpdirname)

# Both the directory and its contents have been deleted

It will print the below output:

$ python3 mk-dir-temp-file.py
Created temporary directory: /var/folders/l7/80bx27yx3hx_0_p1_qtjyyd40000gn/T/tmpn_ke7_rk

Create a Secure Temporary File and Directory

By using mkstemp(), we can create a temporary file in the most secure manner possible. The temporary file created using this method is readable and writable only by the creating user ID. We can pass prefix and suffix arguments to add prefix and suffix to the created file name. By default, it opens the file in binary mode. To open it in text mode, we can pass text=True as an argument to the function. Unlike TemporaryFile(), the file created by mkstemp() doesn't get deleted automatically after closing it.

As you can see in the example below, the user is responsible for deleting the file.

import tempfile
import os

temp_directory = tempfile.TemporaryDirectory()

print("Directory name:", temp_directory)

os.removedirs(temp_directory)

$ python3 mk-secure-dir-temp-file.py
Directory name: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T/tmpf8f6xc53

Similar to mkstemp(), we can create a temporary directory in the most secure manner possible using the mkdtemp() method. And again, like mkstemp(), it also supports prefix and suffix arguments for adding a prefix and suffix to the directory name.

Conclusion

In this article we have learned different ways to create temporary files and directories in Python. You can use temp files in any Python program you want. But just make sure to delete it if the particular method used doesn't automatically delete it on its own. Also keep in mind that behavior may different between operating systems, like the output directory names and file names.

All of these functions we have explained above work with many different arguments, although we have not covered in detail what type of arguments each function takes. If you want to learn more about the tempfile module, you should check out the Python 3 official documentation.