Sigmoid
Squeezes any value into a smooth range between zero and one
- Sigmoid takes any value, no matter how large or how small, and squeezes the result into the range between zero and one.
- Near the middle it reacts sharply to small changes, but toward either end it barely reacts at all.
- Because the result sits between zero and one, it reads like "how likely," which is why it's still common at the point where a model decides yes or no.
- In the middle of a stack of layers, its dull behavior at both ends erases the learning signal, which is why it's rarely used there anymore.
- When the answer is one of several options rather than yes or no, models use a rule that weighs every candidate against each other at once instead.
Contents
1The analogy
Leave your phone's screen brightness on automatic and walk outside at noon. However hard the sun beats down, the screen won't get any brighter past its maximum. Walk into a pitch-dark room, on the other hand, and the screen doesn't go fully black either — it settles at some faint minimum glow. The interesting part is what happens in between. At an in-between brightness, like curtains half open and half closed, even a small change in the light makes the screen noticeably brighter or dimmer. Sigmoid moves the same way. No matter how large the input, it never crosses the upper limit; no matter how small, it never drops below the lower one — and it's only in the middle that it responds with any real sensitivity.
2In detail
Pouring values onto a ruler with fixed ends
Values moving around inside a neural network have no set range. Weight and sum up enough values from the previous layer and you'll get very large numbers and very negative ones alike. Sigmoid takes whatever comes in and always places it at some point between zero and one. However large the positive input, the result only approaches one without ever reaching it; however large the negative input, it only approaches zero without ever touching it.
Order is preserved. A larger input still produces a larger output. It's the comparison of size that stays intact while only the scale gets moved onto a narrower ruler, which makes it convenient for comparing values that came from very different places on the same footing.
Sharp in the middle, dull at both ends
The character of this rule shifts depending on where you are. Near the middle, a small nudge to the input produces a big swing in the output. Move toward either end, though, and doubling or tripling the input barely changes the output at all.
That's an advantage at the point where a decision gets made. Borderline cases get split carefully, while clear-cut cases get nailed down firmly. It's also natural not to bother distinguishing between very strong evidence and slightly stronger evidence — pushing an already-certain judgment even further certain doesn't buy you much.
But in the middle of a network, where the job is simply to carry values along, this same dullness becomes a flaw. Even a modestly large value gets shoved straight to one end or the other, so signals that were meaningfully different get flattened into nearly the same value.
A result that reads like likelihood
Because the result lands between zero and one, people naturally read it as something like a probability. That's exactly why sigmoid sits at the very last step for problems where the answer is one of two things: spam or not, defective or not. A result of 0.8 gets read as leaning that way; something near 0.5 gets treated as uncertain, or handed off to a person.
Things change when the answer is one of several options. Apply sigmoid separately to each candidate and the results won't add up to one, which makes them hard to weigh against each other. For that case, models use a different rule that considers every candidate together and divides among them at once. Where several labels can apply at the same time instead, though, giving each one its own separate sigmoid is actually the right call.
Why it got pushed out of the middle layers
Sigmoid used to sit in the middle of networks too. The problem was that dullness at both ends. A network learns by sending word of how wrong it was back from the output toward the input, and at each layer this rule's slope gets multiplied in along the way. The slope near either end is close to zero, and even at the steepest point, right in the middle, it's still well under one.
What happens when you multiply in a small value, layer after layer? Cross even ten layers and the signal coming back has nearly vanished. The early layers never receive anything to learn from and stay stuck at their starting values. That's the backdrop for why a much simpler, single-threshold rule took over sigmoid's old spot in the middle of deep networks.
3More precisely
Sigmoid is also called the logistic function. Drawn out, it's a wide, flattened S-curve, symmetric above and below its center. When the input is exactly zero, the output is exactly 0.5, and that's also the point where it changes fastest.
The analogy breaks down in places. Screen brightness stops and holds once it hits its maximum, but sigmoid never actually touches one or zero — it only keeps getting closer forever. Unlike a screen, which moves in fixed steps, sigmoid is smooth everywhere, which is exactly why a slope can be measured at any point along it — that smoothness is what makes it usable for learning in the first place. One more thing: landing between zero and one doesn't by itself make a value a well-calibrated probability. Getting it to actually match real-world hit rates takes a separate calibration step, not unlike how screen brightness isn't quite the same scale as brightness the way your eyes perceive it.
The name itself is worth a note too. Sigmoid just means S-shaped, and several other curves share that same rough shape without being the exact same function — the logistic curve described here is simply the one that became the default.
4Try it yourself
5Common misconceptions
It's easy to think whatever sigmoid outputs is an accurate probability, but actually without calibration, a result of 0.9 doesn't mean it's right nine times out of ten.
It's easy to think the output reaches exactly zero or one at the extremes, but actually it only ever keeps getting closer and never actually touches either one.
It's easy to think sigmoid isn't used anymore at all, but actually it was only pushed out of the middle layers — it's still the standard choice at the final step of a yes-or-no decision.
7One-line summary
In shortSigmoid squeezes any input into the range between zero and one and reacts sharply only near the middle, and today it's used mainly at the final step of problems where the answer is one of two things.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02