You’ve probably come across Kotlin’s sequences at one time or another. Maybe you’ve heard that they can process data more efficiently than normal collections. But have you ever wondered exactly what they do, how they achieve the efficiency, or when you should use them?

In this article series, we’re going to cover just about everything there is to know about them!

• In this article, we’ll walk through a story that will help us visualize the difference between sequences and normal collections.
• Then, in upcoming articles, we’ll look inside sequences, uncovering some fascinating characteristics in order to understand them like never before!

Are you ready? Our adventure begins at the local crayon factory!

The Crayon Factory

The crayon factory is full of colorful personalities, but today, tension is waxing as they aren’t able to keep up with demand.

“Listen up, team!” began Marge, the manager. “As most of you know, our latest product is called the Fire Crayon Set. It’s a small box of crayons that includes just the colors that you’d normally associate with fire - like reds, oranges, and yellows. Demand is really heating up, so we’ve got to find ways to process them faster.

“Here’s how it works today - we start off with a big box of unlabeled crayons of different colors, and we have to end up with crayons that…

• “…are the right color
• “…have labels on them, and …
• “…fit inside the Fire Crayon Set box (it fits five crayons).”

With that, she rolled out a diagram that shows how the crayons are currently processed:

Marge said to the team, “As you can see, this process is very inefficient. How can we improve it?”

Coding the Crayon Process

While our friends at the crayon factory are pondering the process, let’s code it up in Kotlin!

We’ll start with a simple data class to represent the crayons. The label will be a nullable property, because the crayons start off without them, and are labeled later in the process.

data class Crayon(
    val color: String, 
    val label: String? = null
)

Next, we’ll create our big box of crayons. Lots of colors!

val BigBoxOfCrayons = setOf(
    Crayon("marigold"),
    Crayon("lime"),
    Crayon("red"),
    Crayon("yellow"),
    Crayon("blue"),
    Crayon("orange"),
    Crayon("grape"),
    Crayon("white"),
    Crayon("red-violet"),
    Crayon("pond"),
    Crayon("cinnamon"),
    Crayon("neon lightning"),
    Crayon("metal"),
    Crayon("violet"),
    Crayon("charcoal"),
    Crayon("brick"),
    Crayon("green"),
    Crayon("silver")
)

And finally, we’ll make a small set that tells us which colors are allowed to be included in the Fire Crayon Set:

val includedColors =
    setOf("brick", "marigold", "neon lightning", "orange", "red", "red-violet", "white", "yellow")

And with that, we’re ready to start processing those crayons! Remember, the flow was:

1. Start with the big box
2. Filter out colors that don’t apply
3. Label the crayons
4. Take just the first five crayons
5. Collect them into their final box

Thanks to the collection operations in Kotlin’s standard library, we can model this five-step process in just five lines:

val fireSet = BigBoxOfCrayons               // 1. Start with the big box
    .filter { it.color in includedColors }  // 2. Filter out colors that don't apply
    .map { it.copy(label = "New!") }        // 3. Label the crayons
    .take(5)                                // 4. Take just the first five crayons
    .toSet()                                // 5. Collect them into their final box

When you run this code, you’ll get a set of Crayon objects that make up the Fire Set.

Here’s how the code above maps to the steps in the process.

Functions like filter(), map(), and take() are often called collection operations. Each one operates on a collection, applying some kind of transformation to it in order to produce another collection.

When we hook together multiple collection operations in a row like this, we call the whole thing a collection operation chain.

As you can see, this code is very declarative - we didn’t have to manually instantiate collection objects or iterate over items. Instead, we just declared what we wanted done, and Kotlin’s standard library did the rest!

As our friends in the crayon factory discovered, though, there’s actually a lot of waste that happens here. Yes, they started with the big box and ended with the Fire Set box, but in the middle, they wasted three boxes.

Similarly, if you were to peek into the implementation of these collection operations, you’d see ArrayList objects (that is, the “boxes” for our crayons) being created and discarded at each of those processing steps - filter(), map(), and take().

Shoveling data out of one collection and into another can be expensive! In a small example like ours, we might not notice the wasted ArrayList instances affecting performance. But the bigger the data set, and the more collection operations being performed, the greater the penalty that we’re likely to experience!

There’s got to be a more efficient way to process those crayons! Let’s see what our friends at the factory think…

Back at the Factory…

After some time, Marge spoke up again. “So, team, how can we make this process more efficient?”

Neumann, the most recent hire, said, “Here’s an idea - what if we did every relevant operation to a single crayon before moving on to the next?” He rolled out a new sheet and drew up this new diagram:

Neumann continued, “With this flow, when the color is right, we wrap the crayon, put it in the final box, and move onto the next crayon. Once the box has all 5 crayons, we stop. In other words, the new process would flow like this…” He pulled out a marker and traced the flow:

Neumann concluded, “It gets rid of all those extra boxes between steps. Plus, we never bother filtering or wrapping crayons that wouldn’t fit in the final box anyway.”

Feeling confident about the change, Marge declared, “Well, that settles it! Let’s put this plan into action!”

And indeed, by changing the process so that the operations were applied to just one crayon at a time, no boxes were wasted, only the necessary crayons were processed, and our friends were finally able to keep up with demand!

Modeling an Efficient Crayon Process

As we saw at the factory, we can improve the efficiency of the crayon process by updating the flow to apply every relevant operation to just one crayon before moving on to the next.

In order to get Kotlin to use Neumann’s new flow instead of the old flow, all we have to do is make one easy change:

val fireSet = BigBoxOfCrayons
    .asSequence()  // <--------------------- the one easy change
    .filter { it.color in includedColors }
    .map { it.copy(label = "New!") }
    .take(5)
    .toSet()

That’s right! In order to use the more efficient flow, all we had to do was to call asSequence() on the BigBoxOfCrayons before running the operations on it!

Here’s how that code maps to Neumann’s diagram.

It’s critical to understand that, in the sequence operation chain, the operations are applied to the individual crayons in a different order than they are in the standard collection operation chain.

To help visualize and understand that difference, let’s look at one more set of diagrams.

These diagrams will depict the same filter(), map(), and take() chain we’ve used all along. For reference, here’s the code again:

// Collection Operation Chain

val fireSet = BigBoxOfCrayons
    .filter { it.color in includedColors }
    .map { it.copy(label = "New!") }
    .take(5)
    .toSet()

// Sequence Operation Chain

val fireSet = BigBoxOfCrayons
    .asSequence()
    .filter { it.color in includedColors }
    .map { it.copy(label = "New!") }
    .take(5)
    .toSet()

Now, let’s arrange the operations on each crayon into a grid, where the crayon colors are along the horizontal axis and the operations are along the vertical axis.

• The color of the circles represents the color of the crayon being processed.
• The letter in the circle represents the operation being performed on that crayon.
• The line running through the circles represents the order that the operations are executed on each of those crayons.

If you follow those lines with your finger, you’ll notice that:

• In the collection operation chain, the letters are grouped together - the line passes through all the F’s and then all the M’s, and then all the T’s. In other words, all of the filtering happens, then all of the mapping happens, then all of the taking happens.
• On the other hand, in the sequence operation chain, the colors are grouped together - the line passes through all of the marigold circles, then all of the lime circles, then all of the red circles, and so on. This means that each crayon is processed entirely before moving onto the next.

You probably also noticed that there are fewer total individual operations performed in the sequence chain - 18 compared to 31 - because processing stops once the box is full!

So, What Is a Sequence?

Now that we’ve seen what a sequence does, it’s time to define the term.

A sequence is an iterable type that can be operated upon without creating unnecessary intermediate collections, by running all applicable operations on each element before moving onto the next.

In this definition, the word iterable does not refer to the Kotlin type Iterable. It refers to the ability to iterate over the sequence, such as in a for loop. There’s no actual relationship between Sequence and Iterable in the type system, but we’ll explore the amazing similarities between them later in this series.

Summary

Congratulations! You now have a clear understanding of the difference between the way that normal collection operations are processed and the way that sequence operations are processed.

We’ve only scratched the surface, though! In the next article, we’ll cover advanced sequence concepts in order to reveal some fascinating characteristics and a few gotchas. We’ll go beyond understanding what they are. We’ll journey into how they work. In fact, we won’t just use sequence operations - we’ll implement our very own!

See you then!

Update: When I first published this article, there wasn’t any information about sequences in the official Kotlin documentation, but good news! - it’s now been updated to include a whole new section about collections, including a page about sequences. Check it out!

Thanks to Karan Trehan and Jacob Rakidzich for reviewing this article.