What is Next-Token Prediction? — one analogy

1The analogy

Take a grocery receipt and fold it in half so the bottom lines are hidden from view. If the top shows milk, eggs, and bread, you can guess what's on the next line — the shape of the whole shopping trip is already forming in your head.

The nice part of this game is that grading is free. Unfold the paper and the answer is printed right there. No answer key to track down, nobody to ask.

One receipt can also generate a lot of questions. Guess line two from line one, guess line three from the first two lines, and so on down the whole slip — one question for every single line. Turn this guessing game into text instead of groceries, and that's next-token prediction.

2In detail

Scoring likelihoods, not picking one outright

A model never singles out one candidate for the next spot. It scores every candidate in its whole list of pieces — tens of thousands of them — with "here's how likely this one is." After "the weather today is really," something like "nice" scores high, "sunny" scores fairly high, and "square" scores very low.

Then it picks based on those scores. Always taking the top score makes answers stiff and repetitive, so a bit of randomness usually gets mixed in among the higher-scoring candidates. That's why the same question can get a slightly different answer each time.

Guess, then check right away

Training is this guessing game repeated an enormous number of times. The text is shown up to some piece, the model guesses the next one, and that guess gets compared against what the original text actually has there.

When it misses, the numbers inside the model get nudged a tiny bit, so the correct answer scores a little higher next time. One correction is too small to notice, but repeat it billions of times and it adds up.

Notably, no one has to step in by hand. The question comes from the text and so does the answer — any writing on the internet becomes a ready-made workbook.

One piece of writing produces a flood of questions

Sliding the hidden spot over by one turns into a brand-new question. A piece of writing with a thousand pieces yields roughly a thousand questions — nothing about the document goes to waste.

How much comes before also keeps changing. Some questions only get three pieces of lead-in to work with; others get hundreds. Practice on short and long context mixes together naturally.

Difficulty swings wildly by position too. The piece right after "thank" is nearly a given, so almost anyone gets it, while the first piece of a story's ending takes holding onto everything that came before just to make a decent guess. Easy and hard questions sit side by side, and working through them the same way over and over widens what gets handled.

Where guessing turns into ability

Guessing the next piece looks trivial. But getting good at it demands more and more the further it goes. Guessing what follows "the capital of France is" needs geography; guessing what follows "three plus four is" needs arithmetic.

The demands only grow with longer text. Guessing the piece after a novel's last line means holding onto who did what earlier in the story. Chasing this one guessing game just kept dragging in knowledge and reasoning along with it.

Producing an answer runs the same routine. Given a question, one piece is chosen to follow it, that piece is appended, the whole thing is read again, and another piece is chosen — over and over. Translation, summarizing, conversation: the only thing happening inside any of them is this one repeated step.

3More precisely

The model's final layer puts out a score for every entry in its whole piece list. Turning those scores into a set of shares that add up to one gives the probability of the next piece. Training nudges the internal numbers so the probability assigned to the correct piece keeps rising, and the value measuring how far off it is is called the loss.

The analogy breaks down in one important spot. A receipt is fully printed already, so it has exactly one correct answer, while text usually has several reasonable pieces that could come next — plenty of things sound fine after "the weather today is really." So training doesn't push toward memorizing one fixed answer; it spreads weight fairly across candidates that all make sense. The model also only looks ahead one piece at a time rather than planning a whole sentence in advance — where an answer ends gets decided as it's being written. And what gets guessed isn't a whole word but something sliced finer than that, so finishing one ordinary word sometimes takes several predictions strung together in a row.

4Try it yourself

Fill in the Blank (what a language model actually learned) ailearn.space Type a sentence and see a ranked list of how likely each candidate for the next spot is

5Common misconceptions

It's easy to think the model already knows exactly what comes next, but actually it scores every candidate and draws from among them, so the same question can land on a different answer.
It's easy to think guessing the next piece is too simple to support anything hard, but actually getting good at it demands knowledge, arithmetic, and context-tracking, and those abilities grow right alongside it.
It's easy to think a whole answer is finished first and then shown one character at a time, but actually each piece is still being decided the moment it appears on screen.

7One-line summary

In shortNext-token prediction is scoring the likelihood of just the very next piece based only on what came before, and nearly everything a language model does stands on this one repeated step.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02