Generators, Iterators, and Advanced Async Patterns

Generators and async generators are powerful tools that enable lazy, on-demand data production — values are computed only when requested. Generators underpin many advanced patterns: infinite sequences, custom iterables, pausable computations, and the async iteration protocol used by the Streams API, WebSockets, and server-sent events. In this lesson you will master generator functions, the iterator protocol, async generators with for await...of, and the real-world patterns that show why these features are invaluable for data streaming, pagination, and reactive event handling.

Generators and the Iterator Protocol #

Async Generators and for await…of #

Generator Use Cases #

Basic Example #

// ── Basic generator ───────────────────────────────────────────────────────
function* counter(start = 0) {
    while (true) {
        yield start++;
    }
}

const gen = counter(1);
console.log(gen.next());   // { value: 1, done: false }
console.log(gen.next());   // { value: 2, done: false }
console.log(gen.next());   // { value: 3, done: false }

// ── Range generator — lazy sequence ──────────────────────────────────────
function* range(start, end, step = 1) {
    for (let i = start; i < end; i += step) {
        yield i;
    }
}

console.log([...range(0, 10, 2)]);   // [0, 2, 4, 6, 8]

for (const n of range(1, 6)) {
    console.log(n);   // 1, 2, 3, 4, 5
}

// ── Finite generator with return value ───────────────────────────────────
function* processItems(items) {
    let processed = 0;
    for (const item of items) {
        yield item;   // pause after each item
        processed++;
    }
    return processed;   // return value on done
}

const processor = processItems(['a', 'b', 'c']);
console.log(processor.next());    // { value: 'a', done: false }
console.log(processor.next());    // { value: 'b', done: false }
console.log(processor.next());    // { value: 'c', done: false }
console.log(processor.next());    // { value: 3, done: true }  ← return value

// ── yield* — delegate to another iterable ────────────────────────────────
function* flatten(arr) {
    for (const item of arr) {
        if (Array.isArray(item)) yield* flatten(item);   // recurse
        else                     yield item;
    }
}

console.log([...flatten([1, [2, [3, 4]], 5])]);   // [1, 2, 3, 4, 5]

// ── Custom iterable object ────────────────────────────────────────────────
class InfiniteCounter {
    constructor(start = 0, step = 1) {
        this.current = start;
        this.step    = step;
    }

    [Symbol.iterator]() {
        return {
            current: this.current,
            step:    this.step,
            next() {
                return { value: (this.current += this.step), done: false };
            }
        };
    }

    *take(n) {
        let count = 0;
        for (const val of this) {
            if (count++ >= n) return;
            yield val;
        }
    }
}

const evens = new InfiniteCounter(0, 2);
console.log([...evens.take(5)]);   // [2, 4, 6, 8, 10]

// ── Async generator — paginated API ──────────────────────────────────────
async function* fetchAllPages(baseUrl) {
    let page = 1;
    let hasMore = true;

    while (hasMore) {
        const response = await fetch(`${baseUrl}?page=${page}&limit=20`);
        if (!response.ok) throw new Error(`HTTP ${response.status}`);

        const { data, meta } = await response.json();
        yield* data;   // yield each item from this page

        hasMore = meta.page < meta.totalPages;
        page++;
    }
}

// Consume all users across all pages — lazily
for await (const user of fetchAllPages('/api/users')) {
    console.log(user.name);   // processes each user as pages load
    if (user.role === 'admin') break;   // stop early — no more fetches
}

// ── Async generator — Server-Sent Events ─────────────────────────────────
async function* streamEvents(url) {
    const response = await fetch(url, {
        headers: { Accept: 'text/event-stream' },
    });

    const reader  = response.body.getReader();
    const decoder = new TextDecoder();

    try {
        while (true) {
            const { value, done } = await reader.read();
            if (done) break;

            const text = decoder.decode(value, { stream: true });
            const lines = text.split('

').filter(Boolean);

            for (const line of lines) {
                const data = line.replace(/^data: /, '').trim();
                if (data === '[DONE]') return;
                try { yield JSON.parse(data); } catch {}
            }
        }
    } finally {
        reader.cancel();   // cleanup when iteration stops
    }
}

// Stream AI response tokens
for await (const chunk of streamEvents('/api/ai/stream')) {
    output.textContent += chunk.delta;
}

How It Works #

Step 1 — Calling a Generator Function Returns an Iterator #

Calling counter() does not run any code — it creates and returns a generator object (which is an iterator). The body only runs when you call .next(). The generator runs until it hits a yield, pauses, and returns { value: yieldedValue, done: false }. When the function body ends (or hits return), done becomes true.

Step 2 — yield Is a Two-Way Channel #

yield value sends value out to the caller. But the expression const x = yield value means that when the caller calls gen.next(someValue), x inside the generator receives someValue. This two-way communication makes generators powerful for coroutines and cooperative multitasking.

Step 3 — for…of and Spread Work With Any Iterable #

Any object implementing the iterator protocol (or having [Symbol.iterator]) works with for...of, spread [...gen], destructuring, Array.from, and Promise.all. Generators automatically implement this protocol — every generator function returns an iterable iterator.

Step 4 — Async Generators Combine Generators and Promises #

An async function* can both await Promises and yield values. Each yield in an async generator produces a Promise that resolves when the value is ready. for await...of automatically awaits each yielded Promise, making the consumption code read sequentially even though each iteration involves async I/O.

Step 5 — try/finally Ensures Resource Cleanup #

When a for...of or for await...of loop exits early (via break, return, or an exception), the iterator’s return() method is called. In a generator, this triggers the finally block. This is where you close open connections, cancel pending requests, and remove event listeners — ensuring resources are always released regardless of how the loop exits.

Real-World Example: Streaming Data Pipeline #

// streaming-pipeline.js

// Composable lazy pipeline stages
function* map(iterable, fn) {
    for (const item of iterable) yield fn(item);
}

function* filter(iterable, fn) {
    for (const item of iterable) { if (fn(item)) yield item; }
}

function* take(iterable, n) {
    let count = 0;
    for (const item of iterable) {
        if (count++ >= n) return;
        yield item;
    }
}

function* chunk(iterable, size) {
    let batch = [];
    for (const item of iterable) {
        batch.push(item);
        if (batch.length === size) { yield batch; batch = []; }
    }
    if (batch.length) yield batch;
}

// Build lazy pipelines — nothing computed until iterated
function* naturals(start = 1) {
    while (true) yield start++;
}

const pipeline = take(
    filter(
        map(naturals(), n => n * n),     // square each number
        n => n % 2 === 0                  // keep even squares
    ),
    5                                     // take first 5
);

console.log([...pipeline]);   // [4, 16, 36, 64, 100]

// Async pipeline — transform a stream of JSON objects
async function* transformStream(asyncIterable) {
    for await (const raw of asyncIterable) {
        // Transform each item as it arrives
        yield {
            ...raw,
            fullName:  `${raw.firstName} ${raw.lastName}`,
            createdAt: new Date(raw.createdAt),
        };
    }
}

// Process in batches of 10
async function processBatches(iterable, batchSize) {
    let batch = [];
    for await (const item of iterable) {
        batch.push(item);
        if (batch.length === batchSize) {
            await saveBatch(batch);
            batch = [];
        }
    }
    if (batch.length) await saveBatch(batch);
}

// Compose: fetch all pages → transform → batch save
await processBatches(
    transformStream(fetchAllPages('/api/users')),
    10
);

Common Mistakes #

Mistake 1 — Spreading an infinite generator #

❌ Wrong — hangs or crashes with out-of-memory:

function* naturals() { let n = 0; while(true) yield n++; }
const all = [...naturals()];  // tries to collect infinite values!

✅ Correct — use take() or a for…of with a break condition:

const first10 = [...take(naturals(), 10)];  // safe

Mistake 2 — Not using try/finally for resource cleanup #

❌ Wrong — reader left open if loop breaks early:

async function* streamData(url) {
    const reader = (await fetch(url)).body.getReader();
    while (true) {
        const { value, done } = await reader.read();
        if (done) break;
        yield value;
        // If caller breaks early — reader never cancelled!
    }
}

✅ Correct — always clean up in finally:

async function* streamData(url) {
    const reader = (await fetch(url)).body.getReader();
    try {
        while (true) {
            const { value, done } = await reader.read();
            if (done) break;
            yield value;
        }
    } finally {
        reader.cancel();  // always runs — even on early break
    }
}

Mistake 3 — Using for…of on an async generator instead of for await…of #

❌ Wrong — yields Promises, not resolved values:

for (const item of asyncGenerator()) {
    console.log(item);  // item is a Promise, not the value!
}

✅ Correct — use for await…of for async generators:

for await (const item of asyncGenerator()) {
    console.log(item);  // resolved value
}

▶ Try It Yourself

Quick Reference #