JavaScript iterators

I occassionally need to generate an array of N values in JavaScript. I like to write a range() function for this, inspired by Python's range():

function range(n) {
	const numbers = [];
	for (let i = 0; i < n; i++) {
		numbers.push(i);
	}
	return numbers;
}

Then I use it this way to generate an array of whatever N values I want:

const items = range(10).map((i) => `Item ${i}`);
//=> ['Item 0', 'Item 1', 'Item 2', ..., 'Item 9']

While it's usually not a big deal, it's slightly disturbing that this range() creates an unnecessary intermediate array. Python's range() avoids this by returning an iterator instead^[1]:

# `range()` doesn't create a list, so there's no intermediate
# list involved when we generate a list of 10 items as follows.
# Note that this `[a for b in c]` syntax is roughly equivalent
# to `c.map((b) => a)` in JS.
items = [f"Item {i}" for i in range(10)]

Let's do something similar in JS.

You can think of an iterator as an object that serves as a loop over a sequence of items. Unlike a regular loop that automatically iterates over the sequence, an iterator's loop is idle by default. This loop advances only when you ask it for the next item in the sequence.

In JS, an iterator is any object with a next() method that returns an object with two properties: value and done. value refers to the next item in the iterator's sequence, and done indicates whether the iterator has already reached the end of its sequence. Here's a range that returns an iterator:

function range(n) {
	let i = 0;
	return {
		next() {
			const item = i;
			if (item < n) {
				i++;
				return { value: item, done: false };
			}
			return { value: undefined, done: true };
		},
	};
}

This range() allows us step through the numbers 0 to N-1 thus:

const iter = range(10);

iter.next(); //=> { value: 0, done: false }
iter.next(); //=> { value: 1, done: false }
// 7 `iter.next()` calls later...
iter.next(); //=> { value: 9, done: false }
iter.next(); //=> { value: undefined, done: true }

Observe that each call to next() advances the iterator until there's no value left for the iterator to yield. Also observe that we can't reuse the iterator, since the iterator keeps moving forward and never backward. In a sense, next() "consumes" the iterator.

Right now, we can't write range(10).map(...), because the range() iterator doesn't have a map() method. Let's give it one that'll consume the iterator and return an array of the mapped values:

function range(n) {
	let i = 0;
	return {
		next() {
			// ...
		},

		map(fn) {
			const result = [];
			while (true) {
				const { done, value } = this.next();
				if (done) {
					return result;
				} else {
					result.push(fn(value));
				}
			}
		},
	};
}

Now, we can generate N items without an intermediate array:^[2]

const items = range(10).map((i) => `Item ${i}`);
//=> ['Item 0', 'Item 1', 'Item 2', ..., 'Item 9']

What if we just want to collect the range() iterator's values into an array, so that we get an array of numbers 0 to N-1?

In Python, we can do this:

numbers = list(range(10))
#=> [0, 1, 2, ..., 9]

The JavaScript equivalent would be:

const numbers = Array.from(range(10));
//=> [] ??

This doesn't work as intended, because Array.from() expects an iterable argument, but the range() iterator is not iterable.

An object in JS is iterable if it implements a special [Symbol.iterator]() method^[3], which returns an iterator over the object. Arrays and strings are iterable, for example, because they have this method.

Let's make the range() iterator iterable. We can do so simply by returning the iterator itself from the [Symbol.iterator]() method:

function range(n) {
	let i = 0;
	return {
		next() {
			// ...
		},

		map(fn) {
			// ...
		},

		[Symbol.iterator]() {
			return this;
		},
	};
}

Now this works:

const numbers = Array.from(range(10));
//=> [0, 1, 2, ..., 9]

You can imagine that Array.from() works something like this:

Array.from = function (iterable) {
	const arr = [];
	const iterator = iterable[Symbol.iterator]();
	while (true) {
		const { done, value } = iterator.next();
		if (done) {
			return arr;
		} else {
			arr.push(value);
		}
	}
};

There are other language constructs besides Array.from() that expect an iterable. The spread syntax is one, and so is the for...of loop:

const numbers = [...range(10)];
//=> [0, 1, 2, ..., 9]

for (const i of range(10)) {
	console.log(i);
}
// The above logs:
// 0
// 1
// 2
// ...
// 9

Creating an iterator as described so far is verbose. We can omit the map() and [Symbol.iterator]() methods if we upgrade our iterator to an instance of the built-in Iterator class, because the class already defines those methods:

function range(n) {
	let i = 0;
	// `Iterator.from()` upgrades the plain iterator to an
	// `Iterator` instance.
	return Iterator.from({
		next() {
			const item = i;
			if (item < n) {
				i++;
				return { value: item, done: false };
			}
			return { value: undefined, done: true };
		},
	});
}

Unlike our custom map(), the built-in map() returns an Iterator instance that yields the mapped values. So, to collect the mapped values in an array, we need to call Array.from():

const items = Array.from(range(10).map((i) => `Item ${i}`));
//=> ['Item 0', 'Item 1', 'Item 2', ..., 'Item 9']

The Iterator class also provides a toArray() method for convenience:

const items = range(10)
	.map((i) => `Item ${i}`)
	.toArray();
//=> ['Item 0', 'Item 1', 'Item 2', ..., 'Item 9']

We can simplify the range() definition further by rewriting it as an equivalent generator function:

function* range(n) {
	for (let i = 0; i < n; i++) {
		yield i;
	}
}

const items = range(10)
	.map((i) => `Item ${i}`)
	.toArray();
//=> ['Item 0', 'Item 1', 'Item 2', ..., 'Item 9']

The function* syntax defines a generator function, and the result of calling the function is a kind of Iterator object called a generator object.

When you call next() on the generator object, the body of the generator function executes up to the next yield expression, then the body is suspended after the yield, and next() returns the yielded value.

When you call next() subsequently, the function body resumes execution, and once again executes up to the next yield expression, before being suspended again, with next() returning the yielded value.

This continues until the function body returns, at which point next() returns { done: true, value: undefined }.

Beyond generating a sequence of N items, iterators are useful for manipulating collections efficiently. As an example, let's say we have an array of objects, and we want to filter it down to the first 3 objects that meet a certain condition. We can do so using the usual array methods:

// Assume `items` is an array of objects.
const firstMatching3 = items
	.filter((item) => item.status === "...")
	.slice(0, 3);

This works fine, but it creates an intermediate array (with filter()) to get to the final result. We can eliminate this by using iterators:

// `items.values()` returns an iterator over the array's elements.
// It's equivalent to `items[Symbol.iterator]()`.
const firstMatching3 = items
	.values()
	.filter((item) => item.status === "...")
	.take(3)
	.toArray();

Here we're using the builtin filter() and take() methods of the Iterator class. You can imagine that they are implemented as follows:

class Iterator {
	filter(fn) {
		const iter = this;
		return Iterator.from({
			next() {
				while (true) {
					const { done, value } = iter.next();
					if (done || fn(value)) {
						return { done, value };
					}
				}
			},
		});
	}

	take(n) {
		const iter = this;
		let i = 0;
		return Iterator.from({
			next() {
				if (i < n) {
					i++;
					return iter.next();
				}
				return { done: true, value: undefined };
			},
		});
	}
}

Or if you prefer the generator syntax:

class Iterator {
	// We write a generator method by prefixing the name with `*`.
	*filter(fn) {
		for (const value of this) {
			if (fn(value)) {
				yield value;
			}
		}
	}

	*take(n) {
		for (let i = 0; i < n; i++) {
			const { done, value } = this.next();
			if (done) {
				return;
			} else {
				yield value;
			}
		}
	}
}

With the definitions above, observe that the iterator version of firstMatching3 works this way:

1. toArray() builds an array by repeatedly calling next() on the take() iterator until the iterator is exhausted. On each call:
   1. The take() iterator calls next() on the filter() iterator.
      1. The filter() iterator calls next() repeatedly on the values() iterator until it reaches the next element that meets the filter condition. Then it yields that element.

The iterator version uses less memory and is potentially faster. If items contains 100 elements and the first 3 elements meet the filter condition, the iterator version will iterate over only those first 3, and will never iterate over the rest of the elements. The array version will however iterate over all 100 elements just to build the filter() array.

Iterators have been in JavaScript as far back as ES6, but the helper methods described so far (map(), filter(), Iterator.from(), etc) are actually new. They were recently accepted into the language by TC39 (JS's governing committee) and will be included in ES2025, the next edition of the JavaScript spec. They are already available in Chromium-based browsers and Firefox, so you can try them out if you're on the latest versions of these browsers.

One thing that excites me is that these new methods make working with built-in iterators a bit more convenient. For example, before these methods, if you had a Map object and wanted to find a value matching a particular condition, you'd have to do something like this:

// Suppose `users` is a `Map` of usernames to user objects.
// Then `users.values()` is an iterator over the `Map` values.
// When iterators had no `find()` method, you'd have to
// collect the values into an array first.
const found = [...users.values()].find((user) => user.email === "...");

Now this is possible and more convenient:

// Note that an iterator's `find()` method returns the
// first matching element or undefined, if there's none.
// It doesn't return an iterator.
const found = users.values().find((user) => user.email === "...");

Credit: Thanks to Ibrahim Adeshina for reviewing drafts of this post.