Benchmark

A benchmark isn't just one set of problems

What gets called a benchmark usually bundles three things: the set of problems to solve, a scoring method that defines what counts as correct, and running conditions, how many times each question gets asked and in what format.

Change any one of the three and the score shifts. Ask the same questions but score the single first answer versus the best of several attempts, and the gap between the two can be wide. That's why reporting a score properly means reporting the conditions along with it.

What it measures is the whole story

There are hundreds of benchmarks out there, and each one measures something different. One measures pulling a fact out of a long passage, another measures reasoning through several steps to reach an answer, another actually runs code and checks whether it passes.

So "the score is high" tells you nothing by itself. It only means something once you know which benchmark it's high on. A battery that held up longest in the cold and one that delivered a big burst briefly are good at different things.

Picking the right benchmark for your work is simple: read a handful of its actual problems yourself. If they resemble what you're trying to get the AI to do, the score is worth something. If they don't, that score belongs to someone else's problem.

When problems go public, answers slip in

Benchmarks are usually published so many people can use them together. But once published, the problems and their answers scatter across the internet, and can end up swept into the material an AI trains on.

When that happens, it's the same as seeing the exam questions beforehand. The model isn't reasoning its way to the answer, it's recalling one it has seen before. The score comes out high, and the moment the question is rephrased even slightly, it collapses. This is called data leakage.

That's why it's increasingly common to grade without publishing the answers, or to swap in fresh problems on a regular schedule.

Scores go stale over time

A brand-new benchmark starts out with low scores across the board, since almost nobody can solve it well yet. A few years on, several candidates land near the top together, and eventually everyone converges close to a perfect score.

Once scores pile up near the ceiling, the benchmark can no longer rank anyone. It's like a test rig with conditions so loose that any product placed on it scores near perfect. That's when a harder or differently angled benchmark takes over.

What a benchmark can't tell you

A benchmark scores a fixed format of problem in a fixed way, so anything that doesn't fit that format never gets measured at all: whether an answer reads well, whether the model admits what it doesn't know, what habits show up after long use.

Speed and cost matter a great deal in an actual service too, and those usually sit outside a benchmark as well. That's why benchmark scores are safest used to narrow down candidates, leaving the final call to results measured on your own data.