Softmax
A calculation that turns several scores into shares totaling 100%
- Softmax divides up several scattered scores so they add up to 100 percent.
- One candidate's score alone can't settle its share — every other candidate's score has to be weighed too.
- The order stays exactly the same after dividing. Whoever scored higher still gets the bigger share.
- It doesn't carry the gap between scores over as-is — it widens it before dividing. A small lead can turn into a large share.
- The resulting percentages aren't a measure of how correct the model is — only the ratio among candidates.
Contents
1The analogy
A commercial building gets one combined water bill. Each shop has its own meter, so the building manager writes down every reading, adds them all up, works out what share of the total each shop used, and splits the bill by that share.
A meter reading alone doesn't settle what a shop owes. The readings from every other shop have to be added in too before a share appears. Even if your own reading is identical to last month's, your share shrinks in a month when the shop next door used a lot more water.
Softmax is this division. It takes the raw score the model wrote down for every candidate and splits it into shares within a 100 percent total. That raw score, the meter reading here, is called a logit.
2In detail
One candidate alone can't settle it
Before choosing the next piece, a model scores every candidate. This score is a raw, unscaled number — looked at alone, there's no way to tell if it's high or low.
Softmax works on the whole set of candidates at once. It adds up every score into a total, then hands each candidate its share of that total. So a candidate's share depends not just on its own score but on what score every other candidate received too.
Weak competitors mean the same score wins a bigger share; when everyone scores about the same, the share splits thin. This is also why each individual share shrinks as the number of candidates grows.
The order stays put, only the scale changes
Dividing up shares this way never touches the ranking. Whoever led on the scoreboard still leads once turned into a share. This calculation isn't meant to set an order — it's meant to move things onto a scale a person can read and compare.
Turning scores into percentages makes a lot possible. Candidates below some cutoff can be trimmed out, an answer can be randomly picked according to probability to add variety, and measuring how far off a model's guess was from the right answer during training is easier in this percentage form too.
The gap gets widened before dividing
The "soft" in the name means it doesn't hand everything to the single top score. Instead of giving first place 100 percent and everyone else zero, it still leaves something for second and third place too, so a close call shows up as a close call in the result.
That said, it doesn't carry the score ratio over untouched either. A gap in scores gets stretched wider once it becomes a share — a slightly ahead candidate takes a fairly large chunk, and a trailing one gets pushed rapidly toward zero. That's why a difference of just a few points on the scoreboard can leave one answer essentially locked in.
Temperature is exactly what controls this stretching. Lower it and the gap widens further, so the top candidate takes nearly everything; raise it and the gap narrows, giving lower candidates a share too.
Not just used at the very last step
The spot where softmax shows up most visibly in a language model is right before choosing the next piece. But it gets used repeatedly inside the model too.
Attention, which decides how much focus to put on which word in a sentence, is a prime example: every word gets a relevance score, and that score is divided within a 100 percent total to become the share of focus. The same calculation decides how much weight goes to which strand when several strands are weighed against each other, as seen elsewhere.
A model that sorts a photo into categories attaches this same calculation at the end too. That's why the percentages shown on a results screen always add up to 100.
3More precisely
Softmax first turns every score into a strictly positive value, then divides each of those values by their sum. That's how it can handle negative scores, why every result lands between 0 and 1, and why the total always comes to exactly 1.
The analogy breaks down in a few places. A water meter's ratio carries straight over into the billing ratio, but softmax inflates the gap between scores before dividing — a shop with twice the reading of another can end up paying ten times as much, so results can look more lopsided than the raw ratio suggests. A meter reading also clearly means an actual amount of water used, but a model's score carries no such underlying reality of its own. So a 90 percent share doesn't mean that answer is right nine times out of ten — it only shows how far ahead that candidate was among the others in the running. Worth remembering too: an answer that isn't in the candidate list at all doesn't even show up as a 0 percent share — it simply never enters the count in the first place.
4Try it yourself
5Common misconceptions
It's easy to think softmax is what picks the answer, but actually it only divides up shares — the actual picking happens in a separate step after.
It's easy to think a 90 percent share means the model is right nine times out of ten, but actually it's just a ratio among candidates, not an accuracy rate.
It's easy to think a score ratio carries straight over into the share, but actually the gap gets inflated before dividing, so the leading candidate takes a much larger share than the raw ratio implies.
7One-line summary
In shortSoftmax gathers scattered raw scores, like meter readings, and divides them so the shares add up to 100 percent.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02