What is Scaling Law? — one analogy

1The analogy

Dump a ton of water on a potted plant hoping it'll grow fast, and the roots rot instead. Move it to a bigger pot alone and it doesn't grow proportionally more, either, and a spot with no sunlight makes the pot size irrelevant no matter how big it is. Pot size, water, and hours of sunlight all have to line up together for it to grow.

Watch it for a few months and you get a feel for how much it grows per month. You can roughly picture how big it'll be by the end of summer, and use that picture to decide ahead of time when to repot it and how much fertilizer to buy.

A scaling law is that same feel, but for models. Grow model size, the amount of text used in training, and the compute time spent training all together, and how much performance improves traces a surprisingly smooth curve. Doubling the pot size doesn't double the height, though — growth doesn't come back in the same proportion you put in.

2In detail

All three have to grow together

There are broadly three things you can grow when scaling up a model: how many values sit inside it, how much text goes into training, and the total amount of compute spent training it.

The three are tightly linked. Show a large model too little text and it just memorizes what it saw, leaving it weak on anything new. Pour endless text into a small model and its container runs out of room past a certain point, and it stops improving. Run short on compute time and even with the first two in place, training finishes only half-done.

That's why growing just one of the three quickly hits a wall. For a while, the emphasis was on growing model size. Later it became clear that, given the same compute budget, keeping the model a bit smaller and showing it far more text instead often works better.

The curve traces smoothly

What's remarkable about this rule is that results aren't jagged. Run experiments doubling or tripling size and data, and the number representing performance traces a gentle curve down rather than jumping around in steps. The same shape holds across several orders of magnitude.

A smooth curve means a predictable one. Run a handful of small experiments, plot a few points, and you can draw a line toward roughly where a much bigger run will land.

The curve also flattens as it goes, though. Early on, a small increase brings a visible improvement; later, a tenfold increase in scale barely nudges things forward. Past a certain point, you're in a stretch where you get less back for what you put in.

So a budget can be planned ahead of time

Training a large model is hard to undo once it's underway. It takes months and an enormous amount of compute, and finding out the setup was wrong only after it's all done is expensive.

A scaling law acts like a map here. Given a fixed amount of available compute, it lets teams calculate ahead of time what ratio to split it between model size and data for the best result. A handful of small experiments' results get used to project a much larger run, and the design gets locked in around that projection.

It works the other way too — start from a target and work backward to the budget needed. Given a target level of performance, roughly what size and how much data would it take? A decision that used to run on instinct turns into a calculation.

What the curve doesn't tell you

What this curve describes is a fairly basic quantity — something like how well the model guesses the next word. Abilities people actually care about, like translating well or working through a long problem to the end, connect to that number only loosely.

It doesn't say when a specific ability will show up, either. Some abilities look like they simply don't work at all until scale crosses a certain line, then suddenly do. The underlying curve stays smooth even as the visible ability seems to jump, which makes it hard to call in advance.

Data quality doesn't show up in the curve either. The same amount of well-curated text produces much better results than raw text does. And the supply of human-written text isn't unlimited, which caps how far this path can be pushed — part of why attention has shifted from growing training toward spending more compute at answer time instead.

3More precisely

A scaling law is an empirical relationship between model size, data volume, compute, and training error. It isn't a proven law of nature — it's a regularity observed repeatedly across many experiments. The value on the vertical axis is usually some measure of how well the model predicts the next word, and it looks close to a straight line once the axes are plotted on a multiplicative scale.

The analogy breaks down in a place worth naming. A potted plant stops growing once it's mature, but this curve has no visible endpoint. Instead there's a floor it can never cross, and the curve just keeps getting closer to that floor. A plant also shows the effect of extra water the same day, but a model only reveals its result once training finishes entirely. Most importantly, a curve improving and an answer being factually correct are two different things — making the scale bigger doesn't make the problem of inventing facts that aren't there go away on its own. A well-tended plant, at least, never invents fruit that isn't really hanging from its branches.

4Try it yourself

5Common misconceptions

It's easy to think making it bigger is enough on its own, but actually growth stalls fast unless data and compute grow alongside it.
It's easy to think a law always holds because it's called a law, but actually it's a pattern observed across experiments, and it can break when conditions change.
It's easy to think the curve tells you when a specific ability will appear, but actually it only shows the trend of a basic underlying quantity.

7One-line summary

In shortA scaling law is the rule that lets you sketch ahead of time how much performance will improve when size, data, and compute all grow together.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02