Model Size
How many numbers a model is carrying
- The '7B' tacked onto a model's name means 7 billion parameters, not a file size.
- Parameters are numbers set during training. The more of them there are, the more the model has memorized, and the more room it has to hold things.
- Producing an answer means loading every one of those numbers into memory at once. So size and required memory are the same thing.
- Whether a model runs on your own computer usually comes down to how much graphics memory you have.
- Round the numbers off coarsely and size drops by half, then by half again, at the cost of some accuracy.
Contents
1The analogy
Picture studying at a library desk where you have to lay every reference book flat on the desk before you can work a single problem. A thin paperback fits comfortably on one side of the desk. A thick multi-volume set won't fit on the desk at all. It doesn't matter whether the book is good — no desk space means you can't even open it.
Model size is the same story. A model carries an enormous number of values it fixed during training, and it flips through those values every time it produces an answer. So the whole set of numbers has to be laid out on the "desk" belonging to whatever device is doing the calculating.
If the desk is too small, there are two options: pick a thinner book, or use a version with smaller print.
2In detail
'B' is a unit for counting
The 7B, 13B, 70B tags next to a model's name are shorthand for how many parameters it has. B stands for billion, so 7B means 7 billion parameters and 70B means 70 billion.
Parameters are numbers the model gradually adjusted while training. They started out meaningless and were tuned as the model saw huge numbers of examples, ending up at their current values. The whole collection of these numbers is everything the model has learned.
So size is less about performance itself and more like the size of the container. It tells you how much room there is — whether something good got put in that room is decided by the data and method used in training.
The count is directly how much room it takes
Storing one number takes some space. In common formats, one number takes about two units of space. Seven billion parameters comes to roughly fourteen billion units of space, which lands around 14 in gigabytes.
There's an easy rule of thumb: double the number in front of the 'B' and you get roughly the gigabytes needed. 7B is around 14, 13B is around 26, 70B is around 140.
Some extra room gets tacked on top of that. The longer a conversation runs, the more temporary space is needed to hold everything said so far. So in practice, you should set aside a few more gigabytes than the rough estimate.
Whether it runs on your computer
The first bottleneck when running a model at home is usually the memory on your graphics card. The graphics card does the calculating, and to calculate, the numbers have to already be loaded onto that card's memory.
Try loading a 14-gigabyte model onto a computer with 8 gigabytes of graphics memory, and it simply won't fit. Force it, and the missing portion gets pulled in on the fly from regular memory or storage, and that back-and-forth takes longer than the actual computing — producing a single character can take several seconds.
So the same model runs smoothly on one computer and is practically unusable on another. Not because it's lacking in ability, but because it doesn't fit on the desk.
Shrinking it down to an abridged edition
A common way to cut size is to write the numbers less precisely. Round a long decimal down to a coarser figure, and the space one number takes drops by half, then by half again.
Shrink the space one number takes from four chunks down to one, and a 14-gigabyte model comes down to around 4 — small enough to fit on an ordinary graphics card at home. Answer quality gets a little blurrier, but in everyday questions the difference is often hard to notice.
There's a limit to how far this shrinking can go, though. Round too coarsely and the calculations start visibly wobbling. Sometimes it's better to use a model that was built small from the start, rather than aggressively shrink a large one.
Bigger isn't always better
A large model knows more and keeps up better with complicated requests. In exchange, it's slower, and needs more hardware and electricity to run.
If the job is well-defined, a small model can actually be the better choice. Things like formatting text a certain way, shortening it, or sorting it into categories — a small model handles these just fine, and the answer arrives much faster.
There's really one rule for picking a size: only as much as the job calls for. Extra size beyond that comes back as waiting time and cost, not better performance.
3More precisely
Model size is usually stated as the number of parameters set through training. The space it actually takes up equals the parameter count times how many units it takes to store one number, plus room set aside for intermediate calculations and conversation history. Common formats use 2 bytes per number, and coarse rounding can bring that down to as little as 0.5 bytes.
The analogy breaks down in one place. You can open a reference book to only the page you need, but an ordinary model runs through every single parameter once for each piece of an answer it produces. Some architectures pick out only the parts they need — a model like that can have a large total size while using a much smaller share of it at any one time. So two models with the same size tag can actually need different hardware. It's also worth remembering that size isn't a ranking of performance. A small model trained well outperforming a large one trained carelessly isn't rare at all.
There's also no single desk. A model often has to be split across several graphics cards at once, and how well that split is arranged can matter almost as much as the raw memory total.
4Try it yourself
5Common misconceptions
It's easy to read the B in 7B as bytes, but actually it's the billions unit for counting parameters.
It's easy to think you only need as much memory as the file size you downloaded, but actually you need extra room on top of that, which grows as a conversation runs longer.
It's easy to think bigger always means smarter, but actually how well something was trained matters just as much as how big it is.
7One-line summary
In shortModel size counts how many parameters a model has, and that count directly decides how much memory it needs, which decides whether it can run on your computer.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02