Encoder-Decoder
A two-part design that reads with one half, writes with the other
- Encoder-decoder is a structure that keeps a reading half and a writing half separate. The front half packs the input into a summary; the back half looks at that summary and produces a new sentence.
- The encoder looks at the whole sentence all at once, front and back together. The decoder writes one piece at a time, moving forward.
- What passes between the two halves isn't the original text — it's a bundle of numbers holding only the meaning. The literal words never cross over.
- It suits jobs where the input and the output don't share the same shape, like translation or summarizing.
- Some models today use only the encoder half, others only the decoder half — whichever fits the job at hand.
Contents
1The analogy
Nobody moves house by shoving an entire room straight into the truck. Clothes from the closet, books from the shelf — everything goes into boxes first, and each box gets labeled with what's inside. At that moment, exactly which drawer a given item came from is lost — all that's left is what there is, and how much of it.
At the new place, the boxes get opened one at a time and everything gets arranged to fit the new rooms. The room sizes and window placements are different, so the old layout can't just get copied over. You look at what's in each box and put it wherever makes sense in this new house.
The half doing the packing is the encoder; the half unpacking at the new place is the decoder. And the boxes crossing between them carry the summarized meaning.
2In detail
The packing side reads the whole sentence at once
The encoder looks at an incoming sentence from start to finish, all at once. It can process the third word while referring to the tenth word with no trouble at all, because the entire text is already sitting right there.
That freedom matters. Whether "bank" means a riverbank or a financial bank often only becomes clear from something later in the sentence. Since the encoder can see ahead, it can settle ambiguity like this in advance.
After the sweep, each piece of the sentence turns into its own bundle of numbers. That bundle carries what role that piece played within the sentence, too — the same way packing a box means not just tossing items in, but also writing down what's inside on the side.
The writing side builds one piece at a time
The decoder has it differently. It has no way to peek at a part it hasn't written yet. So it looks at only two things — the pieces it has written so far, and the boxes the encoder handed over — and picks the next single piece from those. It appends the chosen piece, rereads from the top, and picks the next one after that.
That's why the decoder carries a blindfold blocking anything ahead of it. During training, feeding it the full correct sentence all at once would let it peek at the next piece and cheat, so anything not yet written gets deliberately hidden.
It even signals when it's done on its own. Once it judges there's nothing left to add, it produces the piece that means "stop here" and finishes.
The trick that let it keep glancing back at the boxes
The earliest version of this structure had a frustrating limit — however long the sentence, everything had to get squeezed into one single box. Moving a house full of belongings with just one box meant whatever went in first got crushed under everything else. That's exactly why longer sentences tended to lose track of what came at the start.
The fix was not merging everything into a single box. Keep every one of the per-piece bundles the encoder produced, and let the decoder, every time it writes one piece, pick out and glance at whichever one it needs right then. That mechanism is called attention.
Translating "I read the book my friend gave me yesterday," writing the tense pulls up the box holding "yesterday"; writing the object pulls up the box holding "book." Only opening the box that's actually needed means a long sentence doesn't get any blurrier toward the front.
Today, sometimes only one half gets used
Which half is needed depends on the job. Grasping what a sentence means doesn't need the writing half at all — a model with only the encoder is plenty for jobs like classification or search, where the answer comes out short and clean.
The opposite holds for conversation or writing, where text keeps getting continued — a model with only the decoder is what gets used there. It can treat a question it received as though it were already-written text and just keep going from there. Most conversational models today are built this way.
Structures with both halves still hang around wherever the input and the output are genuinely different — translation or summarizing, where the language or length going in and coming out differ substantially.
3More precisely
The encoder turns input pieces into context-aware bundles of numbers; the decoder takes those bundles together with everything written so far and produces a probability for the next piece. Inside the decoder, the channel that looks at its own output and the channel that looks toward the encoder are kept separate, and the process of hiding what's ahead is called masking.
The analogy breaks down in a place worth naming. Items packed into a moving box stay exactly as they were, but the bundle of numbers the encoder produces doesn't hold the original text — there's no reconstructing it back word for word. And boxes get loaded once and the truck departs, but the decoder glances back at the encoder's side fresh, every single time it produces one more piece — this isn't a handoff that happens just once.
Encoder-decoder isn't the name of a specific model — it's a way of arranging parts. The same arrangement can be built with a recurrent network just as easily as with a transformer, and the two halves don't even need matching sizes; a heavier encoder paired with a lighter decoder is a common trade-off.
4Try it yourself
5Common misconceptions
It's easy to think the encoder stores the sentence like a zip file, but actually it keeps only the meaning in a form that can't be reconstructed, so there's no unzipping it back to the original.
It's easy to think the decoder only receives the encoder's output once, but actually it looks back at the encoder's side fresh every single time it produces one more piece.
It's easy to think every language model has both an encoder and a decoder, but actually a large share of today's conversational models are built with only a decoder.
7One-line summary
In shortEncoder-decoder splits the job into a half that packs the input down to just its meaning, and a half that looks at that meaning and writes a new sentence one piece at a time.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02