Python Iterators

Tutorial 22 of 65 · pythondeck.com Python course

An iterable returns an iterator via iter(); the iterator yields values via next() and raises StopIteration when exhausted. Implement __iter__ and __next__ to write your own.

Iterators are objects that implement __iter__ and __next__, yielding items one at a time until StopIteration. They power for loops, comprehensions, and memory-efficient pipelines.

Separating iterables (can produce iterators) from iterators (single-pass consumption) explains generator behavior and lazy evaluation. That distinction is why reading a large log file line by line keeps memory flat while loading all lines into a list does not.

Protocol: iterable has __iter__; iterator also has __next__.

Built-in iterables: list, dict, file, range; iterators from iter(obj).

Generators: functions with yield return generator iterators.

Generator expressions: (x*x for x in range(10)) like list comp but lazy.

next(it, default) manual advance; StopIteration ends iteration.

itertools module: chain, islice, groupby for combinatorial iteration.

for loops call iter() and next() under the hood. Generator functions pause at yield and resume state between calls—efficient for large or infinite streams.

Iterator exhaustion is permanent; re-create from the source or materialize to list if needed.

Async iterators power async for; debugging may call next(), but application code prefers for.

Consuming a generator twice expecting the same items.

Mutating a collection while iterating its iterator.

Confusing list comprehension [] with generator ()—memory vs laziness.

Using itertools.groupby without sorting key groups first.

Use generators for large or infinite sequences to avoid loading everything into RAM.

Chain processing stages with iterators instead of building intermediate lists.

Document whether your API returns a reusable iterable or a one-shot iterator.

Use enumerate and zip instead of manual next() loops when possible.

Wrap expensive generator pipelines in functions so callers get a clear entry point and docstring.

Re-read the examples below with these ideas in mind; change variable names and inputs to match your own project.

The program below demonstrates built-in. Read the comments on each line, run the code, then change names or values to see how the output shifts.

# Example: Built-in
# Run in the REPL or save as a .py file and execute with python.
it = iter([10, 20, 30])
print(next(it))
print(next(it))
print(next(it))

This sample walks through custom iterator in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.

# Example: Custom iterator
# Run in the REPL or save as a .py file and execute with python.
class Count:
    def __init__(self, n):
        self.i, self.n = 0, n
    def __iter__(self):
        return self
    def __next__(self):
        if self.i >= self.n:
            raise StopIteration
        self.i += 1
        return self.i

print(list(Count(5)))

Here is a hands-on illustration of itertools. Follow the inline comments first; only then execute the snippet and compare the result with what you expected.

# Example: itertools
# Run in the REPL or save as a .py file and execute with python.
from itertools import islice, count
print(list(islice(count(10, 5), 4)))   # [10, 15, 20, 25]

The program below demonstrates manual iterator. Read the comments on each line, run the code, then change names or values to see how the output shifts.

# Iterator protocol: __iter__ returns self, __next__ raises StopIteration
class Countdown:  # custom iterator class
    def __init__(self, start):  # initial value
        self.current = start  # mutable cursor
    def __iter__(self):  # return iterator object
        return self  # this object is its own iterator
    def __next__(self):  # produce next value or stop
        if self.current <= 0:  # termination condition
            raise StopIteration  # signals loop end
        self.current -= 1  # step cursor
        return self.current + 1  # value before decrement

print(list(Countdown(3)))  # [3, 2, 1]

This sample walks through iter function in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.

# iter(callable, sentinel) reads until sentinel value appears
lines = iter(["first", "second", "STOP", "ignored"].__iter__().__next__, "STOP")
for line in lines:  # consumes until STOP
    print("line:", line)  # first and second only
data = {"a": 1, "b": 2}  # mapping
it = iter(data)  # dict iterator yields keys
print(next(it), next(it))  # a b
nums = iter(range(3))  # built-in range iterator
print(list(nums))  # materialize [0,1,2]
print(list(iter(lambda: 0, 1)))  # callable form demo -> [0]

« Python Inheritance All tutorials Python Polymorphism »