Python Circular Imports

What is a Circular Dependency?

A circular dependency occurs when two or more modules depend on each other. This is due to the fact that each module is defined in terms of the other (See Figure 1).

For example:

functionA():
    functionB()

And

functionB():
    functionA()

The code above depicts a fairly obvious circular dependency. functionA() calls functionB(), thus depending on it, and functionB() calls functionA(). This type of circular dependency has some obvious problems, which we'll describe a bit further in the next section.

Figure 1

Problems with Circular Dependencies

Circular dependencies can cause quite a few problems in your code. For example, it may generate tight coupling between modules, and as a consequence, reduced code reusability. This fact also makes the code more difficult to maintain in the long run.

In addition, circular dependencies can be the source of potential failures, such as infinite recursions, memory leaks, and cascade effects. If you're not careful and you have a circular dependency in your code, it can be very difficult to debug the many potential problems it causes.

What is a Circular Import?

Circular importing is a form of circular dependency that is created with the import statement in Python.

For example, let's analyze the following code:

# module1
import module2

def function1():
    module2.function2()

def function3():
    print('Goodbye, World!')

# module2
import module1

def function2():
    print('Hello, World!')
    module1.function3()

# __init__.py

import module1

module1.function1()

When Python imports a module, it checks the module registry to see if the module was already imported. If the module was already registered, Python uses that existing object from cache. The module registry is a table of modules that have been initialized and indexed by module name. This table can be accessed through sys.modules.

If it was not registered, Python finds the module, initializes it if necessary, and executes it in the new module's namespace.

In our example, when Python reaches import module2, it loads and executes it. However, module2 also calls for module1, which in turn defines function1().

The problem occurs when function2() tries to call module1's function3(). Since module1 was loaded first, and in turn loaded module2 before it could reach function3(), that function isn't yet defined and throws an error when called:

$ python __init__.py
Hello, World!
Traceback (most recent call last):
  File "__init__.py", line 3, in <module>
    module1.function1()
  File "/Users/scott/projects/sandbox/python/circular-dep-test/module1/__init__.py", line 5, in function1
    module2.function2()
  File "/Users/scott/projects/sandbox/python/circular-dep-test/module2/__init__.py", line 6, in function2
    module1.function3()
AttributeError: 'module' object has no attribute 'function3'

How to Fix Circular Dependencies

In general, circular imports are the result of bad designs. A deeper analysis of the program could have concluded that the dependency isn't actually required, or that the depended functionality can be moved to different modules that wouldn't contain the circular reference.

A simple solution is that sometimes both modules can just be merged into a single, larger module. The resulting code from our example above would look something like this:

# module 1 & 2

def function1():
    function2()

def function2():
    print('Hello, World!')
    function3()

def function3():
    print('Goodbye, World!')

function1()

However, the merged module may have some unrelated functions (tight coupling) and could become very large if the two modules already have a lot code in them.

So if that doesn't work, another solution could have been to defer the import of module2 to import it only when it is needed. This can be done by placing the import of module2 within the definition of function1():

# module 1

def function1():
    import module2
    module2.function2()

def function3():
    print('Goodbye, World!')

In this case, Python will be able to load all functions in module1 and then load module2 only when needed.

This approach doesn't contradict Python syntax, as the Python documentation says: "It is customary but not required to place all import statements at the beginning of a module (or script, for that matter)".

The Python documentation also says that it is advisable to use import X, instead of other statements, such as from module import *, or from module import a,b,c.

You may also see many code-bases using deferred importing even if there isn't a circular dependency, which speeds up the startup time, so this is not considered bad practice at all (although it may be bad design, depending on your project).

Wrapping up

Circular imports are a specific case of circular references. Generally, they can be resolved with better code design. However, sometimes, the resulting design can contain a large amount of code, or mix unrelated functionalities (tight coupling).

Have you run in to circular imports in your own code? If so, how did you fix it? Let us know in the comments!