Depth Estimation

Distance is missing from a photo

A camera flattens the world down. Everything strung along a single ray of light gets stamped onto the same spot in the frame, whether near or far. Looking at just one photo, there's nothing in the image itself to say whether a palm-sized toy car sits right up close or a real car sits far away.

That's why getting true distance normally needs two eyes. Compare photos taken from two spots and whatever's closer shifts left-right more; that shift amount becomes the distance. Dedicated range hardware measures directly too, by timing how long light takes to bounce back.

Depth estimation does this without that hardware — it's the work of reconstructing lost distance from a single photo already taken. It's less about re-measuring the true answer and more about pinpointing a plausible one.

Gathering clues in the image

The raw material for reconstructing depth isn't different from what a person uses on that foggy road. If one thing blocks another, the blocker is nearer. If the same kind of object shows up small, it's far away. Where two parallel lines converge to a point inside the frame, that's the direction leading away. Ground texture packing tighter reads as farther, and how a shadow falls also splits near from far.

Training runs on piles of photos paired with distances measured alongside them. Photos shot from many angles get matched against distances measured with real hardware, and the system gets trained to guess that value from the photo alone. After enough scenes, a sense builds up for "this look means roughly this far."

So the result is an educated guess, not a measurement. It does well on a new scene when familiar clues show up, and it gets fooled the same way a person would when those clues point the wrong way.

The result is a single depth map

The answer handed back isn't one number — it's an image the same size as the photo. Every location carries one value, usually painted brighter the closer it is. A portrait glowing white with the wall behind it going dark is exactly that black-and-white picture.

That map is what makes cutting a photo into layers possible. Blurring the background while keeping a person sharp, nudging the frame slightly to make a photo shimmer like it has depth, or making a virtual object correctly disappear behind a real pillar — all of that runs on this one layer.

Robots and cars use this map too, as a reference for driver-assist systems. Where safety is on the line, though, a single photo's guess never stands alone — it gets paired with hardware that measures distance directly.

Order, not a step count

This is the limit that trips people up most often. Most depth estimation only reports which side is nearer and by how much, not distance in absolute units. Stretch or shrink an entire scene to twice or half its size and the near-far relationships stay identical — there's no way to tell the difference.

Getting an actual distance in meters needs a reference point. Supplying the camera's lens details, having someone hold up an object of known size once, or feeding in a few hardware-measured distances all calibrate the scale. Before that calibration, the values are just a picture of relative height, like a contour map.

Some scenes are easy to fool

Mirrors and glass are the classic case. A room reflected in a mirror reads in the photo like a real room stretching away, so it sometimes gets drawn as if the wall opened into more space behind it. Scenery through a window glass overlapping with a reflection on that same glass leaves it unclear which distance to record.

A blank white wall, thick fog, or a spot blown out black by backlighting are hard too. The clues are erased there, so the gaps get filled in from what's nearby — and that can push a flat surface out of place or blur a thin edge. Bars, hair, and branches often end up fused into the background too.