How does for loop work in Python ?

In this article, I am going to explain how the for loops work in Python. You will be introduced to a certain protocol that controls for loops and iteration in general.

for loops

You can use for loop to loop over an iterable like so:

In [4]: iterable = [1, 2, 3, 4, 5]

In [5]: for item in iterable:
   ...:     print(item, end=" ")
   ...:
1 2 3 4 5

The iterable in the above example is a list. An iterable in general produces a stream of values. The for loop does the following:

• Assign a value in the stream to item during each iteration
• Execute statements once for each value in the iterable

An important thing to note is the fact that an iterable decides what value to produce.

You can loop over different kinds of objects, not just lists. If you loop over a string, you get the characters in the string:

In [6]: for c in "Guido":
   ...:     print(c, end=" ")
   ...:
G u i d o

If you loop over a dictionary, you get the keys of the dictionary:

In [7]: capitals = {
   ...:     "India": "New Delhi",
   ...:     "Argentina": "Buenos Aires",
   ...:     "Germany": "Berlin"
   ...: }

In [8]: for key in capitals:
   ...:     print(key)
   ...:
India
Argentina
Germany

If you loop over a file, you get the lines of the file:

In [9]: with open("python_wiki.txt") as f:
   ...:     for line in f:
   ...:         print(line, end="")
   ...:
Python is an interpreted, high-level, general-purpose programming language.
Created by Guido van Rossum and first released in 1991, Python's design philosophy emphasizes code readability with its notable use of significant whitespace.
Its language constructs and object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects.
Python is dynamically typed and garbage-collected.
It supports multiple programming paradigms, including procedural, object-oriented, and functional programming.
Python is often described as a "batteries included" language due to its comprehensive standard library.

The iterator protocol

The reason why you can iterate over different objects is that there is a specific protocol called the iterator protocol which controls the iteration in Python.

In [1]: iterable = [1, 2, 3, 4, 5]

In [2]: iterator = iter(iterable) # Step 1 

In [3]: next(iterator)  # Step 2
Out[3]: 1

In [4]: next(iterator)
Out[4]: 2

In [5]: next(iterator)
Out[5]: 3

In [6]: next(iterator)
Out[6]: 4

In [7]: next(iterator)
Out[7]: 5

In [8]: next(iterator)
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-8-4ce711c44abc> in <module>
----> 1 next(iterator)

StopIteration:   # Step 3

The above code illustrates the iterator protocol:

• Step 1: Any iterable passed to the built-in iter() function produces an iterator object.
• Step 2: When an iterator object is passed to the built-in next() function, it yields the 'next' item in the iterable.
• Step 3: If there is no next item, a StopIteration exception will be raised.

When we loop over any iterable (using a for loop, for instance), the above iterator protocol steps come into action.

Simulating the for loop using the iterator protocol

In [13]: iterable = [1, 2, 3, 4, 5]

In [14]: for number in iterable:
    ...:     print(number, end=" ") # statement
    ...:
1 2 3 4 5

By now, you must be very familiar with the above for loop. Let's try to simulate this for loop using the iterator protocol.

In [18]: iterator = iter(iterable) # Step 1: Get the iterator object

In [19]: while 1:
    ...:     try:
    ...:         item = next(iterator)  # Step 2: Get next item from the iterable
    ...:     except StopIteration: # Step 3: No more items
    ...:         break
    ...:     print(item, end=" ") # statement
    ...:
1 2 3 4 5

The above code shows how we can simulate the for loop by manually running the iterator protocol. The for loop in Python is automatically doing what we were doing manually: calling iter to get an iterator and then calling next over and over until a StopIteration exception is raised.

Implicit loops

Many operations in Python consume an iterable object by using the iterator protocol implicitly.

1. Constructors

list(iterable), tuple(iterable), set(iterable), dict(iterable)

2. Reductions

sum(iterable), min(iterable), max(iterable)

3. Unpacking

item1, item2, item3 = iterable

4. Containment checking

item in iterable

All these operations pull the values from the iterable using the iterator protocol one at a time and consume them. They loop over the iterable without having an explicit for loops.

Conclusion

In this post, we have seen how the for loop and in general iteration works under the covers in Python.

Many things in Python is based on the iterator protocol and learning it is critical to understand many concepts in Python. When you start appreciating the iterator protocol, you will think about Python from a different perspective and paves the way for more advanced learning in Python.

I hope, you have gained valuable insights on how for loops work in Python through this post. I encourage you to watch this PyCon talk by Ned Batchelder to get a deeper understanding of the concept and more.

That's it, readers, until next time! Keep coding Python!