Using __slots__ to Store Object Data in Python

Introduction

In Python, every object instance comes pre-built with standard functions and attributes. For example, Python uses a dictionary to store an object's instance attributes. This has many benefits, like allowing us to add new attributes at runtime. However, this convenience comes at a cost.

Dictionaries can consume a fair chunk of memory, especially if we have many instance objects with a large number of attributes. If the performance and memory efficiency of code are critical, we can trade the convenience of dictionaries for __slots__.

In this tutorial, we will look at what __slots__ are and how to use them in Python. We'll also discuss the tradeoffs for using __slots__, and look at their performance when compared to typical classes that store their instance attributes with dictionaries.

What Are _slots_ and How to Use Them?

Slots are class variables that can be assigned a string, an iterable, or a sequence of strings of the instance variable names. When using slots, you name an object's instance variables up front, losing the ability to add them dynamically.

An object instance using slots does not have a built-in dictionary. As a result, more space is saved and accessing attributes is faster.

Let's see it in action. Consider this regular class:

class CharacterWithoutSlots():
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location

without_slots = character_without_slots('Fred Flinstone', 'Bedrock')
print(without_slots.__dict__)  # Print the arguments

In the above snippet:

• organization is a class variable
• name and location are instance variables (note the keyword self in front of them)

While every object instance of the class is created, a dynamic dictionary is allocated under the attribute name as __dict__ which includes all of an object's writable attributes. The output of the above code snippet is:

{'name': 'Fred Flinstone', 'location': 'Bedrock'}

This can be pictorially represented as:

Now, let's see how we can implement this class using slots:

class CharacterWithSlots():
    __slots__ = ["name", "location"]
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location

with_slots = CharacterWithSlots('Fred Flinstone', 'Bedrock')
print(with_slots.__dict__)

In the above snippet:

• organization is a class variable
• name and location are instance variables
• The keyword __slots__ is a class variable holding the list of the instance variables (name and location)

Running that code will give us this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'character_without_slots' object has no attribute '__dict__'

That's right! Object instances of classes with slots do not have a __dict__ attribute. Behind the scenes, instead of storing the instance variables in a dictionary, the values are mapped with the index locations as shown in the figure below:

While there's no __dict__ attribute, you still access the object properties as you would typically do:

print(with_slots.name)         # Fred Flinstone
print(with_slots.location)     # Bedrock
print(with_slots.organization) # Slate Rock and Gravel Company

Slots were created purely for performance improvements as stated by Guido in his authoritative blog post.

Let's see if they outperform standard classes.

Efficiency and Velocity of Slots

We are going to compare objects instantiated with slots to objects instantiated with dictionaries with two tests. Our first test will look at how they allocate memory. Our second test will look at their runtimes.

This benchmarking for memory and runtime is done on Python 3.8.5 using the modules tracemalloc for memory allocation tracing and timeit for the runtime evaluation.

Results may vary on your personal computer:

import tracemalloc
import timeit

# The following `Benchmark` class benchmarks the
# memory consumed by the objects with and without slots
class Benchmark:
    def __enter__(self):
        self.allocated_memory = None
        tracemalloc.start()
        return self

    def __exit__(self, exec_type, exec_value, exec_traceback):
        present, _ = tracemalloc.get_traced_memory()
        tracemalloc.stop()
        self.allocated_memory = present


# The class under evaluation. The following class
# has no slots initialized
class CharacterWithoutSlots():
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location


# The class under evaluation. The following class
# has slots initialized as a class variable
class CharacterWithSlots():
    __slots__ = ["name", "location"]
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location


# The following `calculate_memory` function creates the object for the
# evaluated `class_` argument corresponding to the class and finds the
# memory used
def calculate_memory(class_, number_of_times):
    with Benchmark() as b:
        _ = [class_("Barney", "Bedrock") for x in range(number_of_times)]
    return b.allocated_memory / (1024 * 1024)


# The following `calculate_runtime` function creates the object for the
# evaluated `class_` argument corresponding to the class and finds the
# runtime involved
def calculate_runtime(class_, number_of_times):
    timer = timeit.Timer("instance.name; instance.location",
                         setup="instance = class_('Barney', 'Bedrock')",
                         globals={'class_': class_})
    return timer.timeit(number=number_of_times)


if __name__ == "__main__":
    number_of_runs = 100000   # Alter the number of runs for the class here

    without_slots_bytes = calculate_memory(
        CharacterWithoutSlots, number_of_runs)
    print(f"Without slots Memory Usage: {without_slots_bytes} MiB")

    with_slots_bytes = calculate_memory(CharacterWithSlots, number_of_runs)
    print(f"With slots Memory Usage: {with_slots_bytes} MiB")

    without_slots_seconds = calculate_runtime(
        CharacterWithoutSlots, number_of_runs)
    print(f"Without slots Runtime: {without_slots_seconds} seconds")

    with_slots_seconds = calculate_runtime(
        CharacterWithSlots, number_of_runs)
    print(f"With slots Runtime: {with_slots_seconds} seconds")

In the above snippet, the calculate_memory() function determines the allocated memory, and the calculate_runtime() function determines the runtime evaluation of the class with slots vs the class without slots.

The results will look something along these lines:

Without slots Memory Usage: 15.283058166503906 MiB
With slots Memory Usage: 5.3642578125 MiB
Without slots Runtime: 0.0068232000012358185 seconds
With slots Runtime: 0.006200600000738632 seconds

It's evident that using __slots__ gives an edge over using dictionaries in size and speed. While the speed difference is not particularly noticeable, the size difference is significant.

Gotchas Using Slots

Before you jump into using slots in all of your classes, there are a few caveats to be mindful of:

1. It can only store attributes defined in the __slots__ class variable. For example, in the following snippet when we try to set an attribute for an instance that is not present in the __slots__ variable, we get an AttributeError:

class character_with_slots():
    __slots__ = ["name", "location"]
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location

with_slots = character_with_slots('Fred Flinstone', 'Bedrock')
with_slots.pet = "dino"

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'character_with_slots' object has no attribute 'pet'

With slots, you need to know all the attributes present in the class and define them in the __slots__ variable.

2. Sub-classes will not follow the __slots__ assignment in the superclass. Let's say your base class has the __slots__ attribute assigned and this is inherited to a subclass, the subclass will have a __dict__ attribute by default.

Consider the following snippet where the object of the subclass is checked if its directory contains the __dict__ attribute and the output turns out to be True:

class CharacterWithSlots():
    __slots__ = ["name", "location"]
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location


class SubCharacterWithSlots(CharacterWithSlots):
    def __init__(self, name, location):
        self.name = name
        self.location = location

sub_object = SubCharacterWithSlots("Barney", "Bedrock")

print('__dict__' in dir(sub_object))

Output:

True

This can be averted by declaring the __slots__ variable one more time for the subclass for all the instance variables present in the subclass. Although this seems redundant, the effort can be weighed against the amount of memory saved:

class CharacterWithSlots():
    __slots__ = ["name", "location"]
    organization = "Slate Rock and Gravel Company"

    def __init__(self, name, location):
        self.name = name
        self.location = location

class SubCharacterWithSlots(CharacterWithSlots):
    __slots__ = ["name", "location", "age"]

    def __init__(self, name, location):
        self.name = name
        self.location = location
        self.age = 40

sub_object = SubCharacterWithSlots("Barney", "Bedrock")

print('__dict__' in dir(sub_object))

Output:

False

Conclusion

In this article, we have learned the basics about the __slots__ attribute, and how classes with slots differ from classes with dictionaries. We also benchmarked those two classes with slots being significantly more memory efficient. Finally, we discussed some known caveats with using slots in classes.

If used in the right places, __slots__ can boost performance and optimize the code in making it more memory efficient.