Bounding Box
The smallest box around an object, and where that box sits
- A bounding box is the smallest box that just wraps an object. Whatever box shows on screen is the result.
- One box is written with four numbers — a top-left position (two numbers) plus a width and a height.
- How well it fits is measured by how much it overlaps the correct box — overlapping area divided by combined area.
- The correct box is drawn by hand. If the standard used to draw it wavers, the model wavers right along with it.
- Boxes are always axis-aligned, so a tilted or curvy object drags along background inside it.
Contents
1The analogy
You're cutting a new piece of window screen mesh to fit a frame. Cut it bigger than the frame and the edges curl and buckle; cut it smaller and a gap opens up for insects to get in. The goal is a size that fits the frame exactly.
Once it's cut, you need to check the fit — and instead of eyeballing it and saying "close enough," there's a way to put a number on it. Take the area where the frame and the cut mesh overlap, and divide it by the area the two of them cover together. A perfect fit gives you 1, and the number drops fast the further off you are. A mesh that's half misaligned struggles to clear even 0.5, however generously you look at it.
The frame measurement everyone's checking against is something someone measured with a tape and wrote down. Whether they measured the outer edge of the frame or just the inner groove changes the passing mark entirely. A bounding box works exactly the same way.
2In detail
One box is four numbers
Writing down a box is simple. Note the top-left corner's position on screen with two numbers, then how far right and how far down it extends with two more, and that's it. A version that gives the center point plus width and height carries the same meaning.
Writing the numbers as a fraction of the screen size is common — 30 percent in from the left, width equal to 20 percent of the screen, that kind of thing. Written that way, the box keeps tracking correctly even if the photo gets scaled up or down.
A box always sits flush with the screen's horizontal and vertical axes. Tilted boxes aren't used. So wrapping a box around an object lying at an angle drags a lot of empty background in with it. That's exactly why the next step up is segmentation.
Measuring fit by overlap
Checking how well a box fits means comparing it against a correct box that a person drew. Divide the overlapping area of the two boxes by the total area they cover between them, and you get a number between 0 and 1. A perfect overlap gives 1; barely touching gives something close to 0.
How high that number has to climb to count as "correct" is also something a person decides. A threshold of 0.5 is common, and 0.5 is looser than it sounds — a box can pass while wrapping only about half the object. Where precision really matters, a stricter threshold like 0.75 gets used, or scores get measured at several thresholds and averaged.
In demos, you'll see the box drawn slightly bigger or smaller than the object — that kind of small mismatch usually clears the threshold with room to spare. It looks off to the eye, but the score often doesn't mind at all.
The correct box is drawn by a person
The correct box a model gets compared against is one a person dragged out by hand, photo by photo, with a mouse. Ambiguous moments keep coming up while doing this. Does a mug's handle count as part of the box? Should a partly cropped object be drawn as if the off-screen part were imagined in? Does a reflection in a window get boxed too?
If that standard shifts from person to person, what the model learns as "correct" shifts right along with it. That's why data teams pin down detailed rules up front and have several people draw the same photo to check how closely they agree. Tracing back why a model draws boxes in odd places often turns up a wavering standard in the training data.
What a box can't hold
From image classification, which puts one label on a whole photo, up to object detection, which boxes each object, up again to segmentation, which cuts out shapes pixel by pixel — the bounding box is the middle rung of that ladder.
Being in the middle brings both an upside and a downside. It's widely used because it gives position and size without making data too heavy to prepare. But a curvy object, objects overlapping each other, or an object lying at an angle all drag a lot of the wrong thing into the box. Wherever you need to erase a background cleanly or measure area precisely, a box falls short and you have to move up a rung.
3More precisely
A bounding box is the smallest axis-aligned rectangle that wraps an object, usually written as a top-left coordinate plus width and height, four values in all. The value that measures overlap between a predicted box and the correct one is intersection area divided by union area; a threshold on that value sorts hits from misses, and scores are averaged per label into one number.
Overlap alone misses some differences, too. Two boxes that don't touch at all score exactly 0 regardless of how far apart they are, so it can't tell a near miss from a wildly wrong guess. Training sometimes uses a separate calculation that patches over this gap.
The window screen analogy breaks down in one place. A screen only ever gets one piece of mesh per frame, but a model puts out hundreds of candidate boxes for the same object before filtering them down. And a window frame's measurement is fixed once and for all, but the correct box for an object in a photo is a wavering standard that shifts a little depending on who drew it. Because the rule for what to wrap differs from one data set to another, scores from models trained on different data sets can be hard to compare directly.
4Try it yourself
5Common misconceptions
It's easy to think everything inside the box is the object, but actually the box is axis-aligned, so a fair amount of background gets mixed in with it.
It's easy to assume an overlap score of 0.5 means it's only half right, but actually the ratio is measured against the combined area, so 0.5 already means a fairly good overlap.
It's easy to think the correct box is a fixed, settled answer, but actually a person set the rules and drew it by hand, so the standard varies from one data set to the next.
7One-line summary
In shortA bounding box writes an object's wrapping box as four numbers, scores it by how much it overlaps the correct box, and that correct box is one a person set rules for and drew by hand.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02