Text Generation
Building text by looking ahead and adding the next piece
- Text generation doesn't picture the whole piece of writing at once — it picks one piece at a time and adds it on.
- It isn't fetching a sentence sitting somewhere out there — it's built fresh, right there, piece by piece.
- Picking a piece involves a bit of randomness, so the same request produces a different piece of writing every time.
- Once a piece is out, it never gets undone. A wrong turn early on drags everything after it along.
- Whatever comes before shapes the result. The prompt is the very first groove cut.
Contents
1The analogy
Bring a lock to a locksmith and the smith clamps in a blank key and starts cutting. The first notch gets cut, then it's tested against the lock to see where it catches, which sets the depth of the next notch, then tested again to set the one after that. The finished key wasn't pulled from a drawer — it was built, right there, one notch at a time.
There's one rule to this process: a notch already cut can't be uncut. Cut the third notch too deep and no amount of care on the later notches saves it — the key just won't turn. The only fix is clamping in a fresh blank and starting over.
Bring the same lock in twice and you won't get an identical key either time. The hand cutting it is never perfectly steady from one try to the next.
2In detail
It looks ahead and decides the next slot
An AI doesn't finish composing an answer in its head and then copy it out. It looks at every piece placed so far, picks a single next piece, attaches it, then looks at the whole thing again and picks the one after that. This repeats until it's done.
Text streaming across the screen looks that way because a piece really is being produced one at a time. That's also why a longer answer takes proportionally longer — every single piece costs one more round of computation.
Each candidate piece gets ranked by how plausible it seems at that spot. After "the weather today is," something like "lovely" sits near the top and "refrigerator" sits near the bottom. That ranking comes from having learned, across huge amounts of text, how often one word follows another.
It's built, not fetched
It's easy to assume generated text got pulled from some sentence stored away somewhere. There's no warehouse of sentences inside a model. All that's there is the set of numbers used to pick the next piece.
That's how a sentence nobody has ever written before can come out. Ask for a poem that weaves in your name and your dog's name, and it builds one — even though it's never seen that exact combination anywhere. Making something up from a combination it's never seen is both this technology's strength and its risk.
For the same reason, things that aren't true can come out just as smoothly. What decides a piece isn't "is this true" — it's "does this fit here." A paper title that doesn't exist, or a clause number that was never written, can slide out just as fluently as a real one.
It doesn't come out the same way twice
Always picking the single most likely piece keeps the writing safe, but it also turns stiff and repetitive. So there's usually a bit of shake built into the pick, choosing among the top few candidates rather than always the top one. That's why the same question gets a different answer each time.
How much shake gets used is adjustable. Narrow it and the answer stays steady, which suits anything dealing with facts. Widen it and the wording gets more varied, but the odds of an off-base piece slipping in rise right along with it.
Want the same answer back? It works better to paste that answer back in and ask for a revision than to start the conversation over. A second attempt isn't guaranteed to beat the first.
A piece already out never gets undone
If a person doesn't like an earlier sentence while writing, they delete it and rewrite. Generation doesn't work that way. Once a piece is out, it's used as-is as grounds for picking the next one.
That's how one wrong piece early on ends up dragging the whole piece of writing along. Invent a character that doesn't exist in the first sentence, and every paragraph after keeps building on the premise that character is real. It can't dig itself out of a hole it dug for itself.
If an answer has drifted somewhere strange, starting fresh usually beats asking it to patch what's there. Same as the smith clamping in a new blank.
What comes before decides what comes out
Generation always continues from whatever text sits in front of it. The prompt, the conversation so far, any pasted-in material — all of it becomes the first groove cut. That's why the same model can produce very different results depending on what's placed in front of it.
Give it a sample of the format you want and it follows that format. Set a tone and it continues in that tone. Leave nothing set, and it drifts toward whatever's safest — something that reads like it's been seen before.
Length can be set in advance too, though there's a separate hard cap counted in pieces. Hit that cap and the text just cuts off mid-sentence.
3More precisely
Text generation is the repeated act of scoring every candidate for the next piece, turning those scores into probabilities, and drawing one. Feeding the result back in as input to keep going is called autoregressive generation. Drawing methods range from always taking the single highest-scoring piece to drawing probabilistically among a shortlist of top candidates, and the value that sets how much shake goes into the draw is called temperature. None of this looks ahead more than one piece at a time, which is exactly why a model can paint itself into a corner it has no way of foreseeing.
The analogy has a limit. The locksmith holds the actual lock as a known answer to match, but generation has no lock to match against — instead of confirming correctness, it just leans toward whatever seems plausible. A key is also either right or wrong with nothing in between, while writing can be half right in any number of ways, so it can still read smoothly even with an error buried inside. One more difference: the smith cuts one notch at a time, but a model rereads the entire piece of writing so far every single time it picks a piece.
4Try it yourself
5Common misconceptions
It's easy to think an AI has the whole answer decided before it starts typing it out, but actually each piece gets decided as the next one is picked, so it doesn't know its own ending when it begins.
It's easy to assume it's pasting in a sentence that exists somewhere, but actually there's no warehouse of sentences, so it builds sentences that never existed before, right on the spot.
It's easy to think the same question should get the same answer, but actually there's shake built into the picking process, so it comes out a little different every time.
7One-line summary
In shortText generation repeatedly looks at what's written so far, picks the next piece, and attaches it — carving out new writing on the spot rather than fetching it.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02