What is Colorization? — one analogy

1The analogy

Take a black-and-white printed sketch and color it in with colored pencils. The lines are already drawn, so there's nothing new to draw — just deciding which color goes in which area. Blue for the sky, green for the leaves, brown for the dirt.

Nowhere on that sketch does it say "this coat is navy," though. Whatever color you reach for is a guess built from everything you've seen before. Coloring the sky blue is nearly a sure bet; coloring a roof orange is a plausible pick, not necessarily the right one.

Keeping color from bleeding past the lines takes work too. Let green sneak across where leaves meet sky and the whole picture looks messy, so knowing exactly where one region ends and the next begins matters as much as picking the color itself.

2In detail

Training material is easy to come by

Learning to colorize needs pairs of a black-and-white picture and its colored version. That pair is remarkably easy to make — strip the color out of any color photo and you get its black-and-white twin, so one photo becomes both the question and the answer.

That's a big edge over tasks where a person has to label every example by hand. Hundreds of millions of photos online become practice material as-is, so training data never runs short.

The world captured in that material shows through in the results, though. Train mostly on summer photos and leaves come out deep green every time; train mostly on clear days and skies come out blue every time. An overcast gray sky rarely shows up.

Brightness stays put; only color gets layered on

A black-and-white picture keeps only brightness information. Colorization leaves that brightness untouched and builds a separate layer of color to stack on top. Because brightness never gets recalculated, the original's focus and film grain stay exactly as they were.

That makes a bad result easy to undo, too — strip off just the color layer and you're back to the original, or repaint only the part whose color you don't like.

Line art works much the same way. The lines stay put and only the space inside them gets filled, so there's no risk of a line getting mushy. But a broken line leaves that region open, and color leaks out through the gap — closing up the lines before handing a drawing off for coloring makes a real difference in the result.

What it nails and what it misses

Some things have a color that's nearly fixed. Clear sky, grass, tree bark, skin, asphalt — brightness and texture alone narrow the color down enough that these usually come out looking natural.

The trouble is anything that could be any color at all. Clothes, bags, signs, cars, wallpaper — black-and-white alone can't tell you whether something was red or green. Faced with that, the model reaches for whatever color showed up most in its training material, which is why clothing in an old colorized photo so often comes out looking suspiciously plain.

Averaging uncertainly across several color candidates just leaves the whole frame looking sickly yellow. So modern methods commit to one candidate and color it confidently instead of averaging — which is also why the same photo's clothing color can come out different each time you run it.

A few dots of color are enough to spread

If you have a specific color in mind, you can point it out. Mark a spot on a jacket navy and a spot on a wall cream, and that color spreads out to fill the rest of its region. The model already knows where brightness marks one region off from another, so the color fills in without crossing a line.

You can describe it in words, too. Add something like "autumn evening feel" or "muted tones" and the whole color mood leans that way. Working on an old photo, this is sometimes used to bring back a color a family actually remembers.

3More precisely

Colorization splits a picture into a brightness layer and a color layer, taking the brightness layer as input and generating the color layer to go with it. Because only the color layer needs producing, there's less to compute, and combining it back with the original brightness is a light final step. Since there's no single right answer, forcing color into one exact number tends to collapse toward a washed-out yellow average across every plausible candidate. So the standard approach now either turns color into a choice among several bins, or hands the whole job to a generative model that produces a complete, plausible result.

The colored-pencil comparison breaks down in a few places. A pencil colors one region one color, while the model assigns a color to every point on the screen separately, so the same jacket can shade slightly differently between its lit side and its shadowed side. A pencil also lays color over the original lines a little as it goes, while colorization never touches the brightness layer at all. Most of all, a person can know "that mailbox really was red" and color it accordingly — the model has no way to check, and always reaches for the plausible guess.

4Try it yourself

Palette.fm (colorize black-and-white photos) ailearn.space Upload an old black-and-white photo and see whether clothing colors change every time you run it

5Common misconceptions

It's easy to think the original color is somehow still in the black-and-white photo and gets recovered, but actually color was collapsed into brightness and is gone — there's no original left to recover.
It's easy to think a natural-looking result means the color is correct too, but actually only the near-fixed colors tend to be right; things like clothes and signs are plausible guesses.
It's easy to think a color, once applied, is the correct one, but actually running the same photo again can produce a different color.

7One-line summary

In shortColorization is like coloring a black-and-white sketch with colored pencils: the shapes stay put and only the color gets freshly chosen, and that color is a guess, not something recovered.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02