What is Weight? — one analogy

1The analogy

Take the same test, and it doesn't count the same everywhere. A school might grade a course by setting the midterm at 30 percent, the final at 40 percent, and coursework at 30 percent, deciding those shares ahead of time. Gaining ten points on the final moves the overall grade more than gaining ten points on coursework does.

Some items barely count at all. If attendance is worth one percent, acing it barely nudges the final grade. Items that subtract, like lateness or missed submissions, work the opposite way — the more they pile up, the more the grade drops, rather than rising.

That share — how much something counts — is a weight. If the incoming value is a test score, the weight is the share deciding how much that score counts toward the final result. There's one difference: a school sets its grading shares once, at the start of term, but a model's weights adjust themselves after scoring an enormous number of attempts.

2In detail

Every connection carries its own number

A weight belongs to a connection, not to a neuron. If 100 neurons in one layer all feed into a single neuron in the next layer, that one neuron alone needs 100 weights. If the next layer has 100 neurons, that pair of layers alone produces 10,000 of them.

That's why the parameter counts quoted for model size balloon into the billions so quickly. Most of that number is weights, with a smaller remainder being bias values. The reason model files take up so much storage is exactly that they hold every one of these numbers.

The sign decides the direction

When a weight is positive, a larger input pushes the result larger too. When it's negative, the push goes the other way. Only having a way to push toward yes isn't enough to make a judgment — there also has to be a way to pull toward no.

Think of a model deciding whether an email is spam. Certain words push hard toward "spam"; a signal that it came from a known contact pulls the other way. Once the two forces are combined, whatever direction is left over becomes that neuron's judgment.

Larger magnitudes make things more sensitive

When a weight's magnitude is large, even a small wobble in the input sends the output swinging wildly. That's exactly how overfitting can happen — a model that fits its training data closely but falls apart on anything it hasn't seen before.

That's why training deliberately includes mechanisms that discourage weights from growing too large, commonly by penalizing large values. Push in the opposite direction and squeeze every value down near zero, though, and no signal gets through at all. Training is the process of hunting for the right size somewhere between these two extremes.

Training means adjusting these numbers

A freshly built model's weights start out as meaningless random values. Producing answers in that state is naturally a mess. Training measures how far off the answer was, works out which direction and by how much each weight needs to shift to close that gap, and nudges every weight a little. Repeat that step millions of times and the numbers settle into place.

The key is not moving too far in any single step. Move too far and the weights swing wildly to fit whatever example just came in; move too little and it takes forever to get anywhere. The value that controls how far to move is the learning rate.

Once training is done, they stay fixed

Once training ends, the weights lock in place. Talking to a chatbot doesn't change any of the numbers inside the model. The reason the same question can get a slightly different answer each time isn't that the weights shifted — it's that a bit of randomness gets mixed into how the next word gets picked.

Reworking a model means running another round of training. Adjusting just a small piece instead of retraining the whole thing is called fine-tuning, and a widely used variant leaves the original numbers untouched and simply attaches a small bundle of correction numbers alongside them.

3More precisely

A weight is a table of numbers connecting one layer to the next. A layer's computation happens all at once as a matrix multiplication between the bundle of input values and this table. That bulk of multiplication and addition is exactly the kind of work a graphics card excels at, which is why graphics cards get used to train models in the first place.

The grading-share analogy breaks down in a couple of places. A school's grading shares add up to 100 percent, but weights carry no such constraint — they can be negative, and they can exceed one. And while a person setting grading shares knows exactly what those numbers mean, an individual weight carries no meaning a person can read off directly; explaining what any single value attached to any single neuron actually represents is usually difficult.

How values get stored also affects performance. Storing them at lower precision — fewer digits per number — shrinks a model's file size and memory footprint dramatically, a technique called quantization. It trades away a little accuracy in exchange for letting a model run on a laptop or a phone.

4Try it yourself

TensorFlow Playground ailearn.space Run training and watch the connecting lines change in thickness and color — that thickness is the weight

5Common misconceptions

It's easy to think a person sets each weight by hand, but actually they start out random and training adjusts them on its own.
It's easy to think bigger weights make a better model, but actually values that grow too large produce a model that's overly sensitive to its training data.
It's easy to think each individual weight carries some specific meaning, but actually most are numbers no human can read, with meaning scattered across many values at once.

7One-line summary

In shortA weight is the learned multiplier applied to each input, and training is the process of nudging these numbers, bit by bit, toward the values that get answers right.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02