I think the ergonomics of generators is growing on me.

I like the "syntactic sugar" JavaScript's seen over the past decade (arrow functions, template literals, destructuring assignment, etc.). I think it's because most of these features solved real pain points for me (some of which I didn't even know I had). The benefits were clear and there was plenty of opportunity to wield them.

But there are some oddballs in there... like generator functions. They've been around just as long as other key ES2015+ features, but their practicality hasn't exactly caught on. You might not even immediately recognize one:

function* generateAlphabet() {
  yield "a";
  yield "b";
  // ... 
  yield "z";
}

To be fair, I have found them handy at least once. I've written about using them for destructuring an arbitrary number of items on demand. I still love that use case, but I do wonder if there's something more I'm missing out on.

So, I wanted to give them an honest, intentional shake. Maybe after getting a better feel for them, opportunities would pop up. To my surprise, I think I've begun to appreciate both the ergonomics they offer and the mental model they encourage, at least in certain scenarios. I'll try to unwrap where I'm at. First, let’s step back and flesh things out a bit.

The Iterator & Iterable Protocols

Generators won't make much sense unless we cover the two distinct protocols on which they depend: the iterator and iterable protocols. They both deal with the process of producing an indeterminate sequence of values. The latter protocol builds upon the former.

The Iterator Protocol

This one standardizes the shape & behavior of an object that returns a sequence. At bare minimum, any object is an iterator if:

• It exposes a next() method returning an object containing two properties:
  • value: any
  • done: boolean

That's... it. Here's nice, simple one:

const gospelIterator = {
  index: -1,

  next() {
    const gospels = ["Matthew", "Mark", "Luke", "John"];
    this.index++;

    return {
      value: gospels.at(this.index),
      done: this.index + 1 > gospels.length,
    };
  },
};

gospelIterator.next(); // {value: 'Matthew', done: false}
gospelIterator.next(); // {value: 'Mark', done: false}
gospelIterator.next(); // {value: 'Luke', done: false}
gospelIterator.next(); // {value: 'John', done: false}
gospelIterator.next(); // {value: undefined, done: true}

The sequence doesn't have to end, by the way. Infinite iterators are perfectly OK:

const infiniteIterator = {
  count: 0,
  next() {
    this.count++;

    return {
      value: this.count,
      done: false,
    };
  },
};

infiniteGenerator.next(); // Counts forever...

Aside from providing some consistency, this protocol alone may not feel very useful. The iterable protocol should help.

The Iterable Protocol

An object is iterable if it has a [Symbol.iterator]() method returning an iterator object. Every time you've used a for...of loop or destructured an array, you've leveraged this protocol. It's built into common primitives, including String, Array, and Map.

Implementing it yourself allows you to define how a for...of loop behaves. Building on the example from above, this is an iterable object:

const gospelIteratable = {
  [Symbol.iterator]() {
    return {
      index: -1,

      next() {
        const gospels = ["Matthew", "Mark", "Luke", "John"];
        this.index++;

        return {
          value: gospels.at(this.index),
          done: this.index + 1 > gospels.length,
        };
      },
    };
  },
};

Now, both a for...of loop and destructuring will just work:

for (const author of gospelIteratable) {
  console.log(author); // Matthew, Mark, Luke, John
}

console.log([...gospelIteratable]);
// ['Matthew', 'Mark', 'Luke', 'John']

Let's graduate to an example that can't be so easily mimicked with a simple array. Here's one iterating through every leap year after 1900:

function isLeapYear(year) {
  return year % 100 === 0 ? year % 400 === 0 : year % 4 === 0;
}

const leapYears = {
  [Symbol.iterator]() {
    return {
      startYear: 1900,
      currentYear: new Date().getFullYear(),
      next() {
        this.startYear++;

        while (!isLeapYear(this.startYear)) {
          this.startYear++;
        }

        return {
          value: this.startYear,
          done: this.startYear > this.currentYear,
        };
      },
    };
  },
};

for (const leapYear of leapYears) {
  console.log(leapYear);
}

Notice that we don't need to wait for the entire sequence of years to be built ahead of time. All state is stored within the iterable object itself, and the next item is computed on demand. That's worth camping out on some more.

Lazy Evaluation

Lazy evaluation is one of the most-touted benefits of iterables. We don't need every item in the sequence right from the get-go. In certain circumstances, this can be a great way to prevent performance issues.

Look at our leapYears iterable again. If you wanted to stick with a for loop to handle these values but not use an iterable, you might've reached for a pre-built array:

const leapYears = [];
const startYear = 1900;
const currentYear = new Date().getFullYear();

for (let year = startYear + 1; year <= currentYear; year++) {
  if (isLeapYear(year)) {
      leapYears.push(year);
    }
}

for (const leapYear of leapYears) {
  console.log(leapYear);
}

The code is certainly clear and readable (many would say more so than the previous version), but there are trade-offs: we're executing two for loops instead of one, and more importantly, processing the entire list of values up-front. It's a negligible impact for a scenario like this, but could be more taxing for a costly calculation, or for a much larger dataset. Think of something like this:

for (const thing of getExpensiveThings(1000)) {
  // Do something important with thing.
}

Without a custom iterable behind getExpensiveThings(), the entire list of 1,000 items must be built before anything can be done within the loop. The amount of time between executing the script and doing something of value is needlessly large.

In the same vein, it's nice for when you might not need every item in the sequence. Maybe you want to find the first leap year a person experiences by birth year. Once it's identified, there's no reason to continue. If a pre-built array was used, part of it would end up being populated for nothing.

function getFirstLeapYear(birthYear) {
  for (const leapYear of leapYears) {
    if (leapYear >= birthYear) return leapYear;
  }
  
  return null;
}

// Only evaluates leap years up to 1992:
getFirstLeapYear(1989) // 1992

Obviously, the efficiency gains would be more interesting with highly resource-intensive work, but you get the idea. If items aren't needed, no compute is wasted preparing them.

You're in good company if you're irked by how complex it feels to build one of these, by the way, so let's finally jump to generators – a feature built to make all this a little more ergonomic.

Smoothing Over the Protocols w/ Generators

Here's the same iterable from above, but written as a generator function, which returns a generator 0bject.

function* generateGospels() {
  yield "Matthew";
  yield "Mark";
  yield "Luke";
  yield "John";
}

There are two important constructs here: the function* and yield keywords. The former marks it as a generator function, and you can think of the yield keyword as hitting "pause" whenever the generator is asked for the next value.

Under the hood, the same next() method is called here too. Every time it's invoked, it'll move onto the next yield statement (unless there aren't any more).

const generator = generateGospels();

console.log(generator.next()); // {value: 'Matthew', done: false}

And of course, our for...of loop behaves as expected as well:

for (const gospel of generateGospels()) {
  console.log(gospel);
}

// Matthew
// Mark
// Luke
// John

Remember: iterables (and generators) can be infinite, which means you might see something like this in the wild:

function* multipleGenerator(base) {
  let current = base;

  while (true) {
    yield current;

    current += base;
  }
}

Loops like that look scary, but they don’t have to lock up your browser. As long as a yield is stuck between each iteration, everything will be paused when the next value is requested, and execution can go on its merry way.

const multiplier = multipleGenerator(22);

multiplier.next(); // {value: 22, done: false}
multiplier.next(); // {value: 44, done: false}
multiplier.next(); // {value: 66, done: false}

//... no infinite loop!

That said, something worth calling out: generators execute synchronously, so there's still plenty of opportunity to hold up the main thread. Fortunately, there are ways to prevent it. There's also an AsyncGenerator object that may help navigate challenges like this.

Where I've Begun to Appreciate Them

There's no groundbreaking capability here. I'm hard-pressed to find a problem that can't be solved with more common approaches. But as I've started working with them more, I'm coming to appreciate generators more often. A few reasons that come to mind:

They can help reduce tight coupling.

Generators (and all other iterators) shine at encapsulating themselves, including the management of its own state. More & more, I'm noticing how this helps ease the coupling between components I had always blindly made interdependent.

Scenario: when a button is clicked, you want to sequentially show the moving average of some price over the past five years, starting long ago. You only need the average of one window at a time, and you might not even need every possible item in the set (the user might not click through all the way). This would do the job:

let windowStart = 0;

function calculateMovingAverage(values, windowSize) {
  const section = values.slice(windowStart, windowStart + windowSize);

  if (section.length < windowSize) return null;

  return section.reduce((sum, val) => sum + val, 0) / windowSize;
}

loadButton.addEventListener("click", function () {
  const avg = calculateMovingAverage(prices, 5);
  average.innerHTML = `Average: $${avg}`;
  windowStart++;
});

Every time the button is clicked, the next average is rendered to the screen. But it's lame that we need to have a persistent windowStart variable at such a high scope, and I don't feel great about making the event listener responsible for updating that state. I want it exclusively focused on updating the UI.

On top of that, I might want to derive moving averages somewhere else on the page too. That'd be hard with so many things intersecting with everything else. Boundaries are weak and portability is a no-go.

A generator would help remedy this:

function* calculateMovingAverage(values, windowSize) {
  let windowStart = 0;

  while (windowStart <= values.length - 1) {
    const section = values.slice(windowStart, windowStart + windowSize);
    
    yield section.reduce((sum, val) => sum + val, 0) / windowSize;
    
    windowStart++;
  }
}

const generator = calculateMovingAverage(prices, 5);

loadButton.addEventListener("click", function () {
  const { value } = generator.next();
  average.innerHTML = `Average: $${value}`;
});

There are some nice perks:

• The windowStart variable is only exposed where it's needed. No further.
• Since state and logic are self-contained, you could have multiple, distinct generators being used in parallel with no issue.
• Everything’s more focused in purpose. The math + state are left to the generator, and the click handler updates the DOM. Clear boundaries.

I like that model. And we can push it even further. Up until now, the click handler has been the one asking for the next value, directly depending on our generator to provide it. But we can flip that on it's head, allowing the generator to purely provide the ready-to-go values, leaving the click handler to do just something with it. Neither piece needs to know about the intricacies of the other.

for (const value of calculateMovingAverage(prices, 5)) {
  await new Promise((r) => {
    loadButton.addEventListener(
      "click",
      function () {
        average.innerHTML = `Average: $${value}`;
        r();
      },
      { once: true }
    );
  });
}

I can feel the heads turning & noses scrunching. It's not exactly a pattern I'd consider natural, but I respect the fact that it's possible. If the principle of "inversion of control" comes to mind reading through it, it did for me too. There's virtually no interdependence; no need for one component to know about the implementation details of the other. Once the work is done, the listener's cleaned up and control is given back to the generator. My gut says Uncle Bob might at least appreciate the sentiment (if not, please make me the subject of a bathrobe rant 🤞).

They can help avoid little "annoying" things.

I've been surprised by the number of pesky practices I've often had to use to accomplish things in the past. Think: recursion, callbacks, etc. There's nothing wrong with them, but they don't spark joy.

One area in which that's been felt is recurring loops. Think of a dashboard displaying the latest application vitals every second. Features like this can be divided up into two concerns: requesting the data, and rendering it to the UI. There are plenty of options for getting it done:

setInterval()

You could opt for the classic setInterval() – an appropriate choice since its whole purpose is to do things over and over and over (and over).

// ...and over!

function monitorVitals(cb) {
  setInterval(async () => {
    const vitals = await requestVitals();

    cb(vitals);
  }, 1000);
}

monitorVitals((vitals) => {
  console.log("Update the UI...", vitals);
});

But a couple of things might irk you. To keep those two concerns separate, it requires a callback be passed around, potentially triggering deep wounds of "callback hell" from JavaScript's pre-Promise days. On top of that, the interval doesn't care about how long it takes to request the data. The request could end up lasting longer than your interval, causing weird out-of-order issues.

setTimeout()

As an alternative, maybe you'd go with recursion and a Promise-wrapped setTimeout():

async function monitorVitals(cb) {
  const vitals = await requestVitals();

  cb(vitals);

  await new Promise((r) => {
    setTimeout(() => monitorVitals(cb), 1000);
  });
}

monitorVitals((vitals) => {
  console.log("Update the UI...", vitals);
});

That's fine too. But recursion may have killed your great-grandfather in the war and you don’t want to relive that trauma. You’re also still required to pass that callback around.

while(){}

There's also an infinite while loop, broken up by some asynchronous code.

async function monitorVitals(cb) {
  while (true) {
    await new Promise((r) => setTimeout(r, 1000));

    const vitals = await requestVitals();
    cb(vitals);
  }
}

monitorVitals((vitals) => {
  console.log("Update the UI...", vitals);
});

No more recursion, but that callback remains. Again, there's nothing in any of these examples inherently problematic. These are just tiny thorns in one's side. Fortunately, there's another option.

Enter: The Asynchronous Generator

I hinted at this earlier. A regular generator can become an async generator by placing async in its definition. That is stupidly obvious to write, but it's special in another way. Async generators can use a for await loop to run through the sequence of resolved values.

async function* generateVitals() {
  while (true) {
    const result = await requestVitals();

    await new Promise((r) => setTimeout(r, 1000));

    yield result
  }
}

for await (const vitals of generateVitals()) {
  console.log("Update the UI...", vitals);
}

The resulting behavior is the same, but without the emotional triggers. There are no timing risks, no recursion, and no callbacks. Distinct concerns can neatly keep to themselves. All you need to do is handle the sequence.

They can help make exhaustive pagination more efficient.

You've probably done something like this if you've ever needed every item of a paginated resource.

async function fetchAllItems() {
  let currentPage = 1;
  let hasMore = true;
  let items = [];

  while (hasMore) {
    const data = await requestFromApi(currentPage);

    hasMore = data.hasMore;
    currentPage++;
    
    items = items.concat(data.items);
  }

  return items;
}

Few can walk away from that feeling at complete peace with so many auxiliary variables and list stitching going on. Not to mention, you need to wait for that API to be completely exhausted before anything more interesting can occur.

const allItems = await fetchAllItems();

// Won't run until *all* items are fetched.
for (const item of items) {
  // Do stuff.
}

Not the most responsible solution in terms of timing or memory efficiency. We could get around it by refactoring to process items as each page is requested, but we then might suffer from the same issues we've explored so far.

Try out this option instead:

async function* fetchAllItems() {
  let currentPage = 1;

  while (true) {
    const data = await requestFromApi(currentPage);

    if (!data.hasMore) return;

    currentPage++;

    yield data.items;
  }
}

for await (const items of fetchAllItems()) {
  // Do stuff.
}

Fewer auxiliary variables, much less time before any processing can begin, and concerns are still neatly tucked away from each other. Not bad.

They make it really nice to generate sets of items on-the-fly.

I mentioned this in the beginning, but it's so good, I'm gonna sing its praises again. Since generators are iterable, they can be destructured like you would any array. If you need a utility to generate various batches of things, generators make it real simple:

function* getElements(tagName = 'div') {
  while (true) yield document.createElement(tagName);
}

Now, just destructure away, as many times as you like:

const [el1, el2, el3] = getElements('div');

Objectively beautiful. For some more detail on this trick, see the full post.

I Guess We'll See

It's hard to tell if my newfound appreciation for generators will stick around for the long haul (I'm probably still in a bit of a honeymoon phase with them right now).

But even if the enthusiasm dies tomorrow, I'm glad I've got more reps under my belt. Knowing how to use the tool is good and valuable, but being forced to rethink how I tend to approach a problem is even better. A decent ROI.