How to use Python iterators

Python iterators are Python objects with a countable set of elements that execute the iterator protocol using the iter() and next() functions.

What are Python iterators exactly?

Python iterators are a special type of Python object. They consist of a set of countable elements. This means that you can count the elements from an iterator and iterate over every element in a Python iterator.

Iterators vs. iterables

Python iterators and Python iterables are not the same. It is easy to confuse them, however, since they are closely related. Iterables, such as Python lists, are distinguished by the iter() method, which allows users to iterate through them. As a rule of thumb, anything that is on the right side of the loop header when a for loop is executed is an iterable.

l = [1, 2, 3]
for x in l:
	print(x)

As you can see, the list stored in the variable ‘l’ appears after the keyword ‘in’ in the Python for loop. This is the right side of the loop header, making the list an iterable.

An iterable is derived from a Python iterator. The following code example should help make the difference between the two clearer:

string = "test"
iteratorobject = iter(string)
next(iteratorobject)
iter(iteratorobject)

The code stores a Python string with the value ‘test’ in the variable ‘string’. Strings also fall into the category of iterables, as each letter in the string can be iterated through.

Iterating through strings basically just means that strings also support the iter() function. This is visible in the next line of code where a Python iterator is created by executing the iter function with the previous string. This returns a pointer to the string and stores it in the variable ‘iteratorobject’. So, the Python iterator is in a state. We can change this state by executing the next() method on the Python iterator. It moves the pointer by one character so the ‘iteratorobject’ points to the first letter of the string after the function call.

The iter() function call takes the Python iterator as a parameter and returns a reference to it, making it possible to iterate over Python iterators.

Iterators vs. generators

It’s also important to distinguish between iterators and Python generators. While every generator in Python is also an iterator, the reverse is not true. Unlike a generator, an iterator is not necessarily formed with a function that contains a yield expression.

What are Python iterators used for and why are they used?

The main use of iterators is to iterate. The main advantage of using iterators is that they work according to the ‘lazy evaluation’ principle. This means that each element in a Python iterator can be processed individually without having to load the entire data structure into memory. This is especially advantageous when working with large amounts of data, where only one element has to be loaded at any time.

How to create Python iterators

Creating a Python iterator is not difficult. Simply add the iter() and next() functions to a Python object. The code example below shows how to create an iterator that returns even numbers:

class evenNumbers:
	def __iter__(self):
		self.x = 0
		return self
	def __next__(self):
		a = self.x
		self.x += 2
		return a
testobject = evennumbers()
testiterator = iter(testobject)
print(next(testiterator))
print(next(testiterator))
print(next(testiterator))
print(next(testiterator))

To implement a Python iterator, you start by creating a class. The class in this example is called ‘evennumbers’. Within the class, the functions iter() and next() are implemented along with the rules for iteration.

The iter() function returns a reference to the iterator that should contain a sequence of integers starting with 0. The iteration’s logic, which is to output every second number (resulting in only even numbers being displayed) is located in the next() function.

After the class is defined, an object is created and stored in the variable named ‘testobject’. As shown in the previous code example, the object becomes a Python iterator by executing the iter() function. Following this, there are four calls to next() and then the results are shown on the screen. The code section’s output looks like this:

0
2
4
6

How to set a limit in Python iterators

Without a limit, the iterator iterating over even numbers will continue infinitely, just like the set of even numbers itself. This can be prevented by using a StopIteration statement in the else part of an if-else statement in Python. For example, if you want to use a Python iterator to output all even numbers up to and including 100, you can modify the above code example as follows:

class evenNumbers:
	def __iter__(self):
		self.x = 0
		return self
	def __next__(self):
		if self.x <= 100:
			a = self.x
			self.x += 2
			return a	
		else:
			StopIteration
testobject = evennumbers()
testiterator = iter(testobject)
print(next(testiterator))
print(next(testiterator))
print(next(testiterator))
print(next(testiterator))

The only change made to the code is in the next() function. An additional if-else statement has been added. This checks if the current number is less than or equal to 100. If this condition is met, it will continue to iterate through the set of even numbers. If the number however exceeds 100, an error is triggered by StopIteration.