Introduction to Python Iterators

What are Iterators?

An iterator in Python refers to an object that we can iterate upon. The iterator consists of countable values, and it is possible to traverse through these values, one by one.

The iterator simply implements the Python's iterator protocol. The iterator protocol is a Python class which comes with two special methods, namely __iter__() and __next__(). With these two methods, the iterator is able to compute the next value in the iteration.

With iterators, it is easy for us to work with sequences of items in Python. We don't have to allocate computing resources to all the items in the sequence, rather we iterate upon single item at a time which helps us save the memory space.

In this article, we will study how to work with iterators in Python.

Iterable Objects in Python

An iterable is an object capable of returning an iterator. An iterable can represent both finite and infinite data sources. The iterable directly or indirectly implements the two methods: __iter__() and __next__(). The __iter__() method returns the iterator object while the __next__() method helps us traverse the elements in the iterable object.

Examples of iterable objects in Python include Lists, dictionaries, tuples, and sets.

Creating an Iterator

In Python, we create an iterator by implementing the __iter__() and __next__() methods on the object. Consider the following example:

class IterationExample:
    def __iter__(self):
        self.x = 0
        return self

    def __next__(self):
        y = self.x
        self.x += 1
        return y

classinstance = IterationExample()
element = iter(classinstance)

We have created an iterator named element that prints numbers from 0 to N. We first created an instance of the class and we gave it the name classinstance. We then called the iter() built-in method and passed the name of the class instance as the parameter. This creates the iterator object.

Let us now discuss how to use an iterator to actually iterate through the items.

Iterating through an Iterator

The next() method helps us iterate through the elements of an iterator. Let us demonstrate this with the example given above:

class IterationExample:
    def __iter__(self):
        self.x = 0
        return self

    def __next__(self):
        y = self.x
        self.x += 1
        return y

classinstance = IterationExample()
element = iter(classinstance)

print(next(element))
print(next(element))
print(next(element))
print(next(element))
print(next(element))
print(next(element))
print(next(element))
print(next(element))
print(next(element))
print(next(element))

Output

0
1
2
3
4
5
6
7
8
9

In the script above, we called the next() method and passed the name of the iterator element to the method as the parameter. Every time we do this, the iterator moves to the next element in the sequence. Here is another example:

# create a list
list1 = [0, 5, 10, 15]

# create an iterator
element = iter(list1)

## use next() to traverse/iterate through the list elements

# prints first element, 0
print(next(element))

# prints second element, 5
print(next(element))

## next(element) is similar to element.__next__()

# prints third element, 10
print(element.__next__())

# prints fourth element, 15
print(element.__next__())

Output

0
5
10
15

In the script above, we created a list named list1, which has 4 integers. An iterator named element has been created. The next() method has helped us iterate through the list elements.

Iteration with the "for" Loop

The for loop helps us iterate through any object capable of returning an iterator. For example:

# create a list
list1 = [0, 5, 10, 15]

# create an iterator
element = iter(list1)

# iterate with a for loop
for x in element:
    print(x)

Output

0
5
10
15

In the above code, we created a variable named x, which is used to iterate through the iterator element via a for loop.

Infinite Iterators

An infinite iterator is an iterator with infinite number of iterations. We must be extra careful when dealing with infinite iterators. Consider the following example:

class IterationExample:
    def __iter__(self):
        self.x = 0
        return self

    def __next__(self):
        y = self.x
        self.x += 1
        return y

classinstance = IterationExample()
element = iter(classinstance)

for x in element:
    print(x)

The above code will run forever. To stop it, you will have to intervene manually. Here is another example demonstrating how to create an infinite iterator in Python:

class Infinite:
    # Print all even numbers

    def __iter__(self):
        self.x = 0
        return self

    def __next__(self):
        x = self.x
        self.x += 2
        return x

The code should return all even numbers, starting from 0. We can run the code as demonstrated below:

>>> y = iter(Infinite())
>>> next(y)
0
>>> next(y)
2
>>> next(y)
4
>>> next(y)
6
>>> next(y)
8
>>> next(y)
10
>>>

And the above chain can continue forever. This shows that with an infinite iterator, we can have an infinite number of items without having to store all of them in the memory.

In the next section, we will see how we can implement a mechanism to break out of such infinite iterators.

Stopping an Iteration

In the previous section, we saw how to create an infinite iterator in Python. However, iterators are not usually meant for infinite iteration in Python. It is always convenient to implement a terminating condition.

We can stop an iterator from executing forever using the StopIteration statement. We only need to add a terminating condition in the __next__() method which will raise an error once the specified number of iterations has been reached. Here is an example:

class StoppingIteration:
    def __iter__(self):
        self.x = 1
        return self

    def __next__(self):
        if self.x <= 5:
            y = self.x
            self.x += 1
            return y
        else:
            raise StopIteration

classinstance = StoppingIteration()
element = iter(classinstance)

for a in element:
    print(a)

Output

1
2
3
4
5

The execution halts after 5 iterations. This is because of the self.x <= 5: condition added within the __next__() method. If the iterator is called after reaching 5, it will raise StopIteration event. Consider the example given below:

class StoppingIteration:
    def __init__(self, max = 0):
        self.max = max

    def __iter__(self):
        self.x = 1
        return self

    def __next__(self):
        if self.x <= self.max:
            val = 3 ** self.x
            self.x += 1
            return val
        else:
            raise StopIteration

Let us create an iterator then iterate through it:

>>> y = StoppingIteration(3)
>>> z = iter(y)
>>> next(z)
3
>>> next(z)
9
>>> next(z)
27
>>> next(z)
Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    next(z)
  File "C:\Users\admin\iteration.py", line 17, in __next__
    raise StopIteration
StopIteration
>>>

The terminating condition has been implemented in the following section of our code:

if self.x <= self.max:
    val = 3 ** self.x

We passed a value of 3 to the iterator, meaning that the iterator should not iterate beyond 27, that is, 3^3.

Conclusion

Iterators are extremely useful, especially if you need to iterate through a large sequence of items. Iterators allow you to iterate through a sequence of items one at a time without having to load all the items in to memory at once.

In this article, we saw how to create iterators in Python and how to iterate through items in an iterator. We also saw how to create an infinite iterator and how to add terminating condition to an infinite iterator.