Temperature
A dial that widens or narrows the gap between candidates' odds
- Temperature is a dial that widens or narrows the gap in odds among candidates for the next word.
- Turn it down and the strength piles onto the safest word, giving crisp, predictable writing.
- Turn it up and lower-ranked candidates get a chance too, giving uneven writing sprinkled with less common words.
- The dial itself doesn't pick a word. Sampling does the picking — temperature only reshapes things beforehand.
- The ranking among candidates never changes. First place doesn't drop to second — only the gap between them shifts.
Contents
1The analogy
Every home's radiator has a temperature dial. That dial doesn't make new hot water — it only decides how far the water already circulating reaches. Set it low and only the rooms near the boiler warm up while the far room stays cool. Turn it up and warmth reaches even the far corner room, so every room feels warm.
A single dial does something similar when AI picks the next word. The model scores every candidate for "how likely is this word," and lowering temperature pulls the warmth toward the top candidate, leaving the rest effectively cold. Raising it sends warmth out to lower-ranked candidates too, giving them a real chance of being picked. This dial that adjusts the gap between likelihoods is temperature.
No matter how far the dial turns, it never creates a room that wasn't there to begin with. The list of candidates stays exactly as it was — only how the warmth is spread across them changes.
2In detail
The dial itself doesn't do the picking
Producing one piece happens in two steps. First the model scores how likely every plausible next word is. Then one gets drawn according to those scores. That drawing step is sampling.
Temperature steps in between those two. It doesn't leave the scored odds as they are — it widens or narrows the gap first, then hands things off to the drawing step. Like a radiator dial that doesn't make water but only adjusts its flow.
So changing temperature alone changes the result, but it doesn't invent an answer that wasn't possible before. A word whose original odds were near zero still barely turns up even with temperature cranked way up.
Turn it down for crisp, turn it up for uneven
Lower temperature and the gap between the top candidate and the rest widens a lot. Nearly the same word gets drawn every time, so sentences turn out crisp and predictable — ask the same question ten times and similar answers come back. The tradeoff is flatter writing with phrases that repeat often.
Raise temperature and the gap narrows. Second- and third-ranked candidates get drawn often enough to show up, expressions vary, and the same question gets a different answer each time. Useful for pulling several directions of ad copy, or writing fiction.
Push it too far and sentences fall apart. Once words with very low odds start getting drawn, off-topic words creep in and things stop making sense a few sentences later. That's why the range actually used in practice is narrower than it might seem.
What happens at the lowest setting
Drop temperature all the way down and essentially only the top candidate survives. Despite the name "drawing," the single most likely word comes out every time, exactly the same. This gets used when the same question needs the same answer every time.
Cleaning up structured values, filling out a fixed-format document, work that needs point-by-point consistency — this is where it fits. When answers keep changing, whoever's checking them just gets tired.
It's hard to guarantee a fully identical answer every time even here, though. Small differences can creep in from computation order or service-side factors, and even a tiny change earlier in a conversation can send the answer down a different path.
How high or low to set it
There's no fixed answer, but there's a shape to it. Summarizing facts or writing to a strict format calls for low; wanting a few different phrasings calls for the middle; wanting to see unexpected combinations calls for high.
Starting from the default is usually a good idea — most services already tune it to a spot that reads well. Nudge it up if results feel dull, and pull it down if answers wander or facts start slipping.
Moving it a little at a time and running the same question over several tries builds intuition faster than one big jump. Changing just this one dial shows right away how much the answer wobbles.
Top-p is a different dial
A setting often mentioned alongside temperature is top-p sampling. The two do different jobs. Temperature adjusts the gap between candidates; top-p trims the range of candidates available to draw from, cutting off everything below a certain point.
In the heating picture, temperature decides how evenly warmth spreads through the house, while top-p decides which rooms get their door shut ahead of time. However much warmth circulates, a shut room never gets used.
Moving both a lot at once makes it hard to tell which one shifted the result. Usually it's better to leave one at its default and only move the other.
3More precisely
The model puts out a score for every candidate, and those scores go through a calculation that turns them into probabilities. Temperature is the value the whole set of scores is divided by right before that calculation runs. Below one, the score gaps widen and the probability distribution gets sharper; above one, the gaps shrink and it flattens out. The ranking itself never changes at any value.
The analogy breaks down in a few places. A room's temperature rises gradually and stays warm once turned up, but this dial gets reapplied fresh every single time a piece is chosen — no warmth carries over from one letter to the next. And turning up temperature doesn't generate better wording either. The list of candidates was already fixed by the model; the dial only raises the odds that a less common one from that list gets picked. It's closer to a randomness dial than a creativity dial. The range varies by service too — some run from 0 to 1, others allow up to 2, so the same number can feel completely different. Because of that, carrying over a value from another service is riskier than just running a few tries in place and tuning from there.
4Try it yourself
5Common misconceptions
It's easy to think raising temperature makes a model more creative, but actually it only increases how often less common choices get picked, which doesn't sort the odd from the inspired.
It's easy to think temperature at zero always gives the exact same answer, but actually small differences in computation or in earlier conversation can still shift the result.
It's easy to think temperature changes what the model knows, but actually the candidates and their ranking stay the same — only the gap between them shifts.
7One-line summary
In shortTemperature is like a radiator dial deciding whether warmth piles onto the top candidate or spreads out to the far corners, and it never does the actual picking itself.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02