What is Sequence-to-Sequence? — one analogy

1The analogy

Picture taking an order at a sandwich counter. A customer says, "One foot-long turkey club, a side of fries, oh and extra pickles," and the counter never interrupts — it listens all the way to the end. Only once the customer finishes does it jot the whole order down on a single ticket and hand that ticket back to the kitchen.

The kitchen never hears the customer's voice. Working only from the ticket that arrives, it builds the sandwich, fries the fries, and plates the pickles, putting out one item after another. How many words the customer used and how many items come out on the tray have nothing to do with each other. Splitting the listening half from the building half this way is exactly what sequence-to-sequence does.

2In detail

The listening half and the building half are separate

The recurrent network covered earlier took in one word and immediately put out one word. That approach can't handle translation. Word order and word count differ between languages, so a model can't hear the first word and immediately produce a first word in response.

Sequence-to-sequence splits the model into two pieces entirely. The front half only reads the sentence in and produces nothing. The back half does no reading at all — it only produces. The two halves carry separate knobs and are trained together as one system.

Nothing starts until the listening is done

The reading half takes the sentence in one word at a time, continuously updating a running summary. Once the last word has gone in, a single summary bundle is left over from that moment. This bundle stands in for the entire sentence — the whole freight of meaning riding on it.

The building half takes that bundle as its starting point and produces a first word. Then it feeds its own most recent output back into itself to produce the next one. This feedback loop keeps repeating until it's done.

There's also a dedicated marker for knowing when to stop. A signal for "the sentence has ended" and a signal for "start the sentence" are treated like ordinary words and trained right alongside everything else. Once the builder produces the end signal, it stops there.

Why the lengths are free to differ

Because reading and building are completely separated, there's no reason the counts need to match. The reader runs for as many words as it's given; the builder runs until it produces the end signal. Reading four words and producing seven causes no problem at all.

This property opens the door to a much wider range of uses. Condensing a long piece into something short, attaching a long answer to a short question, transcribing speech into text — they all fit inside the same frame. It doesn't matter that what goes in and what comes out are different kinds of things.

When one ticket isn't enough, things leak

The trouble is that there's only one ticket. Whether the order runs three words or thirty, it all has to be squeezed into one bundle of the same fixed size. As the order gets longer, the earliest parts start losing their place.

This showed up clearly in translation. Short sentences came through fine, but as sentences grew longer, entire earlier clauses would go missing or come out garbled. Since the limitation came from the structure itself, making the model bigger didn't fix it.

Where things stand now

The fix was to stop handing over just one ticket and instead let the builder look back at the original text whenever it needed to. Every time it produces one item, it's as if the builder is choosing which part of what the customer said matters right now. That mechanism is attention.

Once attention was added, performance jumped sharply, and it wasn't long before the recurrent structure was stripped out entirely, leaving a model built purely from attention: the transformer. Today's translators and conversational assistants have different parts under the hood, but they still inherit the same backbone — split the listening from the building, and translate between different lengths.

3More precisely

Sequence-to-sequence joins two networks, an encoder and a decoder, into a frame that turns an input sequence of unfixed length into an output sequence of a possibly different length. The encoder's final hidden state becomes a context vector that feeds into the decoder's first state. Early versions built both networks out of recurrent networks or long short-term memory units.

The analogy breaks down in places. A paper ticket carries writing a person can read, but a context vector is a bundle of numbers that can't be read even if you open it up. And while a kitchen can build several plates at once, a decoder has to see what it just produced before it can produce the next item, so output always comes out strictly one piece at a time.

Behavior also differs a little between training and actual use. During training, the correct sentence gets fed in ahead of time so the model learns to predict the next word from it; in actual use, the model feeds its own most recent output back into itself. That's why one early mistake can cascade into a string of mistakes afterward.

4Try it yourself

5Common misconceptions

It's easy to think the input and output counts have to match, but actually letting them differ is the whole point of this structure.
It's easy to think the builder keeps looking at the original text the entire time, but actually the basic version hands the builder nothing but a single summary bundle instead of the original.
It's easy to think this frame disappeared once the transformer arrived, but actually the idea of splitting listening from building carried straight through — only the parts underneath changed.

7One-line summary

In shortSequence-to-sequence reads all the way through into one summary, then builds output one piece at a time from that summary alone, so it can handle inputs and outputs of different lengths.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02