Epoch
One full pass through all the training data
- An epoch is the unit for one complete pass through all the prepared training data, start to finish.
- One pass isn't enough. The values settle into place gradually, over many passes of the same data.
- Even within a single pass, the values get corrected many times, because the data is split into chunks and each chunk earns one step.
- Too few passes and the model undertrains; too many and it memorizes the data outright. There's a sweet spot in between.
- The data gets shuffled at the start of every pass, to keep the model from memorizing the order too.
Contents
1The analogy
Swim a 25-meter lane from one end to the other and back and you've done one lap. That lap is the basic unit of practice. Your form doesn't lock in after just one lap — the feel of catching the water, the rhythm of your breathing, all of it settles in gradually, over many laps.
You're not just coasting through a single lap either. You adjust your wrist angle with every stroke, tweak your kick width along the way. Dozens of small corrections happen inside one lap.
An epoch is that one lap. Once you've gone through the prepared data from start to finish, one lap is done, and you go back to the start for lap two. How many laps to swim gets decided before you ever get in the water.
2In detail
Many steps live inside one lap
Correct the values after every single example and things get too jumpy; wait until the entire dataset has been seen and correct just once, and it's too slow. So the data gets split into chunks of a reasonable size, and the values move one step for every chunk.
Say there are ten thousand examples split into chunks of a hundred — that's a hundred steps in one lap. Ten laps means a thousand steps. That's why progress gets discussed in terms of both lap count and step count together. Two runs with the same lap count but a smaller chunk size will have taken far more steps.
This structure is also why training pauses briefly at the end of every lap to check the score. A set of data held back specifically for this gets pulled out and tested against everything learned so far, and that score gets logged alongside the lap number. Enough of these logs and it's easy to see at a glance whether training is going well.
How many laps to swim
There's no fixed answer. With little data and a big model, dozens of laps might get swum; with an enormous amount of data, sometimes just one lap is enough. Very large language models often have so much data that they don't even finish a single full lap.
Too few laps and the model hasn't finished learning what it could. Too many, and it memorizes the data outright, which actually hurts its score on anything new. In practice, the usual move is to set a generous lap limit and stop once the score starts turning worse.
Shuffling the order every lap
Feed the data in the same order every lap and the model picks up habits tied to that order too — learning whatever type sits near the front first, and whatever sits near the back last, every single time.
So the data gets shuffled and rechunked at the start of every lap. The same data lands in a slightly different combination each time, which means the slope underfoot shifts a bit each lap, cutting the risk of settling into some mediocre spot.
There's another reason: the order data was collected in often carries meaning of its own. Feed it in as-is and the model ends up staring at one category for a stretch, and the values skew that direction during that stretch. A single shuffle wipes out that kind of skew.
Why count laps separately at all
Measuring training progress by clock time makes the number depend on the machine's speed. Counting laps instead shows exactly how many times the data has been seen, which makes runs on completely different machines comparable.
Lap count also becomes the horizontal axis on a score chart. Plot the score on training data and the score on held-back data against the lap number, and it becomes obvious exactly when the two lines start to pull apart. That's where the decision of when to stop comes from.
3More precisely
One epoch means the entire training set has passed through the model exactly once. Counted in steps, that's the number of examples divided by the chunk size. There's also an approach that only sets a step count and never counts epochs at all — when data keeps streaming in continuously, there's no such thing as "the whole set," so the idea of a lap stops making sense. The lap count written into a training setup is often just a ceiling, too, with something else in place to stop training before it's ever reached.
The comparison breaks down in places too. A pool lane has a fixed length, but training data can grow or get filtered partway through. And a person gets tired after many laps, while a model never tires — but keep showing it the same data and it locks in answers that only fit that data. More laps isn't automatically better, which is exactly where this differs from a person's practice. A swimmer also always covers the same water, stroke for stroke, while a model sees the data reshuffled into new chunks every lap, so no two epochs ever move through the values in quite the same order.
4Try it yourself
5Common misconceptions
It's easy to think more epochs always makes a model better, but actually cross a certain point and the score on new data starts dropping instead.
It's easy to think the values get corrected once per epoch, but actually they get corrected many times within a single lap, once per chunk.
It's easy to think the epoch count alone tells you the amount of learning, but actually you need the chunk size and the data size together to know the real step count.
7One-line summary
In shortAn epoch is the unit for one full pass through the prepared data, and how many passes get made decides whether training succeeds or fails.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02