Unit Testing in Python with Unittest

Introduction

In almost all fields, products are thoroughly tested before being released to the market to ensure its quality and that it works as intended.

Medicine, cosmetic products, vehicles, phones, laptops are all tested to ensure that they uphold a certain level of quality that was promised to the consumer. Given the influence and reach of software in our daily lives, it is important that we test our software thoroughly before releasing it to our users to avoid issues coming up when it is in use.

There are various ways and methods of testing our software, and in this article we will concentrate on testing our Python programs using the Unittest framework.

Unit Testing vs Other Forms of Testing

There are various ways to test software which are majorly grouped into functional and non-functional testing.

• Non-functional testing: Meant to verify and check the non-functional aspects of the software such as reliability, security, availability, and scalability. Examples of non-functional testing include load testing and stress testing.
• Functional testing: Involves testing our software against the functional requirements to ensure that it delivers the functionality required. For example, we can test if our shopping platform sends emails to users after placing their orders by simulating that scenario and checking for the email.

Unit testing falls under functional testing alongside integration testing and regression testing.

Unit testing refers to a method of testing where software is broken down into different components (units) and each unit is tested functionally and in isolation from the other units or modules.

A unit here refers to the smallest part of a system that achieves a single function and is testable. The goal of unit testing is to verify that each component of a system performs as expected which in turn confirms that the entire system meets and delivers the functional requirements.

Unit testing is generally performed before integration testing since, in order to verify that parts of a system work well together, we have to first verify that they work as expected individually first. It is also generally carried out by the developers building the individual components during the development process.

Benefits of Unit Testing

Unit testing is beneficial in that it fixes bugs and issues early in the development process and eventually speeds it up.

The cost of fixing bugs identified during unit testing is also low as compared to fixing them during integration testing or while in production.

Unit tests also serve as documentation of the project by defining what each part of the system does through well written and documented tests. When refactoring a system or adding features, unit tests help guard against changes that break the existing functionality.

Unittest Framework

Inspired by the JUnit testing framework for Java, unittest is a testing framework for Python programs that comes bundled with the Python distribution since Python 2.1. It is sometimes referred to as PyUnit. The framework supports the automation and aggregation of tests and common setup and shutdown code for them.

It achieves this and more through the following concepts:

• Test Fixture: Defines the preparation required to the execution of the tests and any actions that need to be done after the conclusion of a test. Fixtures can include database setup and connection, creation of temporary files or directories, and the subsequent cleanup or deletion of the files after the test has been completed.
• Test Case: Refers to the individual test that checks for a specific response in a given scenario with specific inputs.
• Test Suite: Represents an aggregation of test cases that are related and should be executed together.
• Test Runner: Coordinates the execution of the tests and provides the results of the testing process to the user through a graphical user interface, the terminal or a report written to a file.

unittest is not the only testing framework for Python out there, others include Pytest, Robot Framework, Lettuce for BDD, and Behave Framework.

If you're interested in reading more about Test-Driven Development in Python with PyTest, we've got you covered!

Unittest Framework in Action

We are going to explore the unittest framework by building a simple calculator application and writing the tests to verify that it works as expected. We will use the Test-Driven Development process by starting with the tests then implementing the functionality to make the tests pass.

Even though it is a good practice to develop our Python application in a virtual environment, for this example it will not be mandatory since unittest ships with the Python distribution and we will not need any other external packages to build our calculator.

Our calculator will perform simple addition, subtraction, multiplication, and division operations between two integers. These requirements will guide our functional tests using the unittest framework.

We will test the four operations supported by our calculator separately and write the tests for each in a separate test suite since the tests of a particular operation are expected to be executed together. Our test suites will be housed in one file and our calculator in a separate file.

Our calculator will be a SimpleCalculator class with functions to handle the four operations expected of it. Let us begin testing by writing the tests for the addition operation in our test_simple_calculator.py:

import unittest
from simple_calculator import SimpleCalculator

class AdditionTestSuite(unittest.TestCase):
    def setUp(self):
        """ Executed before every test case """
        self.calculator = SimpleCalculator()

    def tearDown(self):
        """ Executed after every test case """
        print("\ntearDown executing after the test case. Result:")

    def test_addition_two_integers(self):
        result = self.calculator.sum(5, 6)
        self.assertEqual(result, 11)

    def test_addition_integer_string(self):
        result = self.calculator.sum(5, "6")
        self.assertEqual(result, "ERROR")

    def test_addition_negative_integers(self):
        result = self.calculator.sum(-5, -6)
        self.assertEqual(result, -11)
        self.assertNotEqual(result, 11)

# Execute all the tests when the file is executed
if __name__ == "__main__":
    unittest.main()

We start by importing the unittest module and creating a test suite(AdditionTestSuite) for the addition operation.

In it, we create a setUp() method that is called before every test case to create our SimpleCalculator object that will be used to perform the calculations.

The tearDown() method is executed after every test case and since we do not have much use for it at the moment, we will just use it to print out the results of each test.

The functions test_addition_two_integers(), test_addition_integer_string() and test_addition_negative_integers() are our test cases. The calculator is expected to add two positive or negative integers and return the sum. When presented with an integer and a string, our calculator is supposed to return an error.

The assertEqual() and assertNotEqual() are functions that are used to validate the output of our calculator. The assertEqual() function checks whether the two values provided are equal, in our case, we expect the sum of 5 and 6 to be 11, so we will compare this to the value returned by our calculator.

If the two values are equal, the test has passed. Other assertion functions offered by unittest include:

• assertTrue(a): Checks whether the expression provided is true
• assertGreater(a, b): Checks whether a is greater than b
• assertNotIn(a, b): Checks whether a is in b
• assertLessEqual(a, b): Checks whether a is less or equal to b
• etc...

A list of these assertions can be found in this cheat sheet.

When we execute the test file, this is the output:

$ python3 test_simple_calulator.py

tearDown executing after the test case. Result:
E
tearDown executing after the test case. Result:
E
tearDown executing after the test case. Result:
E
======================================================================
ERROR: test_addition_integer_string (__main__.AdditionTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_simple_calulator.py", line 22, in test_addition_integer_string
    result = self.calculator.sum(5, "6")
AttributeError: 'SimpleCalculator' object has no attribute 'sum'

======================================================================
ERROR: test_addition_negative_integers (__main__.AdditionTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_simple_calulator.py", line 26, in test_addition_negative_integers
    result = self.calculator.sum(-5, -6)
AttributeError: 'SimpleCalculator' object has no attribute 'sum'

======================================================================
ERROR: test_addition_two_integers (__main__.AdditionTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_simple_calulator.py", line 18, in test_addition_two_integers
    result = self.calculator.sum(5, 6)
AttributeError: 'SimpleCalculator' object has no attribute 'sum'

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (errors=3)

At the top of the output, we can see the execution of the tearDown() function through the printing of the message we specified. This is followed by the letter E and error messages arising from the execution of our tests.

There are three possible outcomes of a test, it can pass, fail, or encounter an error. The unittest framework indicates the three scenarios by using:

• A full-stop (.): Indicates a passing test
• The letter ‘F’: Indicates a failing test
• The letter ‘E’: Indicates an error occured during the execution of the test

In our case, we are seeing the letter E, meaning that our tests encountered errors that occurred when executing our tests. We are receiving errors because we have not yet implemented the addition functionality of our calculator:

class SimpleCalculator:
    def sum(self, a, b):
        """ Function to add two integers """
        return a + b

Our calculator is now ready to add two numbers, but to be sure it will perform as expected, let us remove the tearDown() function from our tests and run our tests once again:

$ python3 test_simple_calulator.py
E..
======================================================================
ERROR: test_addition_integer_string (__main__.AdditionTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_simple_calulator.py", line 22, in test_addition_integer_string
    result = self.calculator.sum(5, "6")
  File "/Users/robley/Desktop/code/python/unittest_demo/src/simple_calculator.py", line 7, in sum
    return a + b
TypeError: unsupported operand type(s) for +: 'int' and 'str'

----------------------------------------------------------------------
Ran 3 tests in 0.002s

FAILED (errors=1)

Our errors have reduced from 3 to just once 1. The report summary on the first line E.. indicates that one test resulted in an error and could not complete execution, and the remaining two passed. To make the first test pass, we have to refactor our sum function as follows:

    def sum(self, a, b):
        if isinstance(a, int) and isinstance(b, int):
            return a + b

When we run our tests one more time:

$ python3 test_simple_calulator.py
F..
======================================================================
FAIL: test_addition_integer_string (__main__.AdditionTestSuite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_simple_calulator.py", line 23, in test_addition_integer_string
    self.assertEqual(result, "ERROR")
AssertionError: None != 'ERROR'

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

This time, our sum function executes to completion but our test fails. This is because we did not return any value when one of the inputs is not an integer. Our assertion compares None to ERROR and since they are not equal, the test fails. To make our test pass we have to return the error in our sum() function:

def sum(self, a, b):
    if isinstance(a, int) and isinstance(b, int):
        return a + b
    else:
        return "ERROR"

And when we run our tests:

$ python3 test_simple_calulator.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

All our tests pass now and we get 3 full-stops to indicate all our 3 tests for the addition functionality are passing. The subtraction, multiplication, and division test suites are also implemented in a similar fashion.

We can also test if an exception is raised. For instance, when a number is divided by zero, the ZeroDivisionError exception is raised. In our DivisionTestSuite, we can confirm whether the exception was raised:

class DivisionTestSuite(unittest.TestCase):
    def setUp(self):
        """ Executed before every test case """
        self.calculator = SimpleCalculator()

    def test_divide_by_zero_exception(self):
        with self.assertRaises(ZeroDivisionError):
            self.calculator.divide(10, 0)

The test_divide_by_zero_exception() will execute the divide(10, 0) function of our calculator and confirm that the exception was indeed raised. We can execute the DivisionTestSuite in isolation, as follows:

$ python3 -m unittest test_simple_calulator.DivisionTestSuite.test_divide_by_zero_exception
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

The full division functionality test suite can found in the gist linked below alongside the test suites for the multiplication and subtraction functionality.

Conclusion

In this article, we have explored the unittest framework and identified the situations where it is used when developing Python programs. The unittest framework, also known as PyUnit, comes with the Python distribution by default as opposed to other testing frameworks. In a TDD-manner, we wrote the tests for a simple calculator, executed the tests and then implemented the functionality to make the tests pass.

The unittest framework provided the functionality to create and group test cases and check the output of our calculator against the expected output to verify that it's working as expected.

The full calculator and test suites can be found here in this gist on GitHub.