What is OCR? — one analogy

1The analogy

The writing carved into an old stone tablet is hard to make out just by looking. So you lay paper over it and tap ink across the surface to lift the strokes — making that rubbing is, itself, the act of reading it.

There's an order to it. First you find which face of the stone actually has writing on it, and lay paper only there, smooth and taut. Lay it on crooked and the strokes stretch or smear, making the finished rubbing hard to read. The ink has to go on evenly too, not heavier on one side.

Once the rubbing is made, you don't read it character by character — you read it line by line. A worn-away character isn't forced; it gets left blank for now, and filled in at the end by following what the surrounding sentences say.

This is exactly what OCR does. Instead of squinting at one faint stroke, it finds the spot, straightens it out, then reads front to back while filling gaps from what's around it.

2In detail

First, find where the text is

A single photo of a storefront sign has far more non-text than text in it, so the first stage isn't reading at all — it's finding, with boxes, where the text is clustered. Skip it and read the whole screen instead, and brick patterns or tree branches get misread as letters.

The unit it looks for is usually a line — finding a whole row of connected characters as one strip works more reliably than hunting character by character. For a table or receipt, finding the cell boundaries too makes it easier to later match each value to the right field.

Once the location is found, only that part gets cropped and passed on — the picture that needs reading shrinks several times over, and computation drops with it.

Straightening things out first

The cropped strip doesn't get read as-is. Text in a photo can be tilted, lit unevenly, or curved like the middle of an open book, so the first move is cleanup: rotate it flat, even out the lighting, and flatten out any curve.

This cleanup changes the result a lot — recognition accuracy on the same document can differ noticeably between a photo shot at an angle and one shot dead straight. That's why document-scanning apps grab the four corners and flatten the frame into a straight rectangle for you.

How lighting gets handled matters too. Apply one single brightness setting to the whole page and a shadowed section of text can vanish entirely; split the picture into small regions with a separate standard for each, and strokes survive even in an unevenly lit photo.

Reading line by line, not character by character

The old approach cut each character out separately and compared its shape — an approach that falls apart the moment characters touch or a stroke breaks, since there's no clean way to even decide where to cut.

Modern methods don't cut at all — they scan a strip left to right and produce characters one after another as they go. Where one character ends isn't decided in advance; the boundary settles naturally as the scan proceeds, which is exactly why this holds up far better against joined-up handwriting or cursive script.

It also looks at neighbors together. When the current stroke is ambiguous, the characters that came before act as a hint — after a digit, the next character is more likely a digit too, and after a letter, an odd symbol out of nowhere is less likely. This judgment is baked right into the recognition itself.

A dictionary and context do the finishing work

The raw read doesn't go straight out the door. Characters that look alike are always trouble — digit 0 and letter O, digit 1 and a vertical stroke.

So there's a stage that fills gaps using a known word list and context. If context suggests a place name, it's checked against a list of place names; a phone field is restricted to digits; a date field filters out impossible values. The more fixed the format, the better this stage works.

Nowadays this cleanup step sometimes gets handed to a language model, feeding in the whole recognized sentence and having it fix anything that reads awkwardly. But since it can also invent plausible-sounding text that wasn't in the original, anything that really matters — numbers, names — needs to be checked back against the source.

Korean has a lot more characters to choose from

The alphabet, upper and lower case combined, is a few dozen characters. Korean's Hangul, built by combining consonants and vowels, runs to thousands of distinct characters, and mixing in Chinese characters pushes the count higher still. Having that many candidates to choose from is a difficulty in itself.

There are also a lot of similar-looking pairs, differing by just one stroke or one final consonant, so a slightly blurred print can easily slip into the wrong character. Vertical writing or an old typeface makes it trickier still.

That's why tools built for Korean gather their own dedicated training data — lots of fonts, sizes, and deliberately blurred samples, built out and trained on at scale.

3More precisely

Performance gets measured two ways: how often a single character is wrong, and how often a whole word is wrong. A word with just one typo still counts as entirely wrong even if character-level accuracy looks fine, so the two numbers can diverge quite a bit — search cares more about the word-level number, faithfully reproducing the original text cares more about the character-level one.

Recognized text carries position information too. Because the box a character came from is stored with it, a translation can be overlaid right on the photo, or a passage traced back to where it came from.

The analogy breaks down in one place. A person reading a rubbing can say "this spot just isn't legible" and leave it blank, but OCR pretty much always hands back an answer. Even at low confidence, it fills in the most plausible character it can, so wrong characters end up mixed in with no obvious sign — which is why the confidence value attached to the result needs checking too, routing low-confidence spots to a person. And where a person works backward from the depth and direction of the remaining strokes, OCR only looks at how closely something resembles shapes it saw in training.

4Try it yourself

5Common misconceptions

It's easy to think you just feed in a photo and text comes straight out, but actually finding the location and cleanup shape the result heavily, so how the photo was taken often decides the accuracy.
It's easy to assume each character gets recognized separately, but actually it scans line by line, taking neighboring characters into account together.
It's easy to believe that if text came out, it must be correct, but actually the most plausible character gets filled in even at low confidence, so numbers and names need to be checked back against the source.

7One-line summary

In shortOCR finds where text sits first, straightens it out, then reads it line by line, and the context-based cleanup at the end is what decides the quality.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02