Iterators and the Iteration Protocol

Python’s for loop works on any iterable — lists, tuples, strings, dicts, files, database cursors, and custom objects. The magic that makes this possible is the iteration protocol: two dunder methods, __iter__ and __next__, that any object can implement to become iterable. Understanding the protocol explains why generators, itertools, and database cursor objects all work seamlessly in for loops — and gives you the knowledge to create your own iterable data structures for FastAPI pagination, streaming, and custom query result types.

Iterables vs Iterators #

# An ITERABLE has __iter__() — returns an iterator
# An ITERATOR has __next__() — returns the next value (and __iter__ returning self)

# Lists are iterables (not iterators)
numbers = [1, 2, 3]
iter_obj = iter(numbers)       # calls numbers.__iter__()
print(type(iter_obj))          # <class 'list_iterator'>

# Call next() to get each value
print(next(iter_obj))   # 1  — calls iter_obj.__next__()
print(next(iter_obj))   # 2
print(next(iter_obj))   # 3
try:
    print(next(iter_obj))   # StopIteration — signals exhaustion
except StopIteration:
    print("Iterator exhausted")

# A for loop does this automatically:
for n in [1, 2, 3]:
    print(n)
# Equivalent to:
_iter = iter([1, 2, 3])
while True:
    try:
        n = next(_iter)
        print(n)
    except StopIteration:
        break

# Iterators are single-use — they cannot be rewound
it = iter([1, 2, 3])
list(it)   # [1, 2, 3]
list(it)   # [] — already exhausted!

# Iterables can be iterated multiple times
lst = [1, 2, 3]
list(lst)   # [1, 2, 3]
list(lst)   # [1, 2, 3] — still works

Implementing a Custom Iterator #

class CountUp:
    """An iterator that counts from start to stop (inclusive)."""

    def __init__(self, start: int, stop: int):
        self.current = start
        self.stop    = stop

    def __iter__(self):
        """Return the iterator object (self, since we implement __next__)."""
        return self

    def __next__(self):
        """Return the next value or raise StopIteration."""
        if self.current > self.stop:
            raise StopIteration
        value = self.current
        self.current += 1
        return value

# Usage
counter = CountUp(1, 5)
for n in counter:
    print(n)   # 1, 2, 3, 4, 5

# Also works with sum(), list(), etc.
print(list(CountUp(1, 5)))   # [1, 2, 3, 4, 5]
print(sum(CountUp(1, 5)))    # 15

# ── Iterable class (separate iterator state) ──────────────────────────────────
class NumberRange:
    """An ITERABLE (not iterator) — can be iterated multiple times."""

    def __init__(self, start: int, stop: int):
        self.start = start
        self.stop  = stop

    def __iter__(self):
        """Return a NEW iterator each time — allows multiple iterations."""
        return CountUp(self.start, self.stop)

r = NumberRange(1, 3)
print(list(r))   # [1, 2, 3]
print(list(r))   # [1, 2, 3] — still works, __iter__ creates new CountUp each time

Built-in Iterables #

# Everything iterable in Python implements __iter__
# Lists, tuples, sets, dicts, strings, files, ranges, generators...

# Check if something is iterable
from collections.abc import Iterable, Iterator

isinstance([1, 2, 3], Iterable)      # True
isinstance("hello",   Iterable)      # True
isinstance(42,        Iterable)      # False

isinstance(iter([1, 2, 3]), Iterator)  # True — list_iterator IS an iterator
isinstance([1, 2, 3],       Iterator)  # False — list is only an iterable

# Common patterns
first = next(iter(some_list), None)   # first item or None if empty
has_any = any(True for _ in iterable)  # True if any item exists
count = sum(1 for _ in iterable)        # count items without building a list

Common Mistakes #

Mistake 1 — Iterating an exhausted iterator a second time #

❌ Wrong — second iteration gives nothing:

it = iter([1, 2, 3])
for x in it: print(x)   # 1, 2, 3
for x in it: print(x)   # nothing — iterator exhausted!

✅ Correct — use the original iterable, or reset:

data = [1, 2, 3]
for x in data: print(x)   # 1, 2, 3
for x in data: print(x)   # 1, 2, 3 ✓ — iterable, not iterator

Mistake 2 — Confusing iter() and next() error handling #

❌ Wrong — no default, raises StopIteration on empty:

first = next(iter([]))   # StopIteration!

✅ Correct — provide a default:

first = next(iter([]), None)   # None ✓ — no exception

Mistake 3 — Testing truthiness of a generator before iterating #

❌ Wrong — generators are always truthy even if they yield nothing:

gen = (x for x in [])
if gen:   # True! Empty generators are still truthy objects
    print("has items")   # wrongly prints!

✅ Correct — use next() with a sentinel to check emptiness:

gen = (x for x in [])
first = next(gen, None)
if first is not None:
    # process first, then continue with gen
    pass

Quick Reference #