This is the first essay in the Evaluation pillar. If you only ever read one model launch post, read this one first — it is the lens you need on the rest of them.

Three things that go wrong

Almost every “our model beats the previous state of the art” claim has one of three problems hiding inside it.

1. Contamination

The most famous English-language benchmark for LLMs is MMLU: 57 subjects, multiple choice, drawn from real exams.1 It is also, by now, almost certainly inside the training data of every frontier model. Web-scale crawls do not respect benchmark licenses; once a benchmark is published it is on Reddit, Stack Exchange, and a dozen study sites within months.

A 2023 paper by Deng and colleagues showed that you can detect this contamination directly — models complete benchmark questions verbatim when given a tiny prefix.2 The original GPT-3 paper already devoted a long appendix to estimating contamination on its own evaluations and discounting scores accordingly.3 Most launch posts skip that step.

2. Saturation

When a benchmark has been around long enough, the headroom disappears. MMLU was designed in 2020 to be hard; by 2024 the top models were within a few points of the human-expert ceiling, which means a one-point improvement could be noise, could be contamination, or could be a real advance — you cannot tell from the number alone.

Benchmarks like GPQA, deliberately built from graduate-level questions that resist web search, exist precisely because the older benchmarks ran out of room.4 But the same dynamic will eat them too, eventually.

3. Teaching to the test

Once a benchmark becomes part of how labs are judged, it becomes part of how labs train. There is nothing inherently dishonest about training on problem types that resemble the benchmark — but it does mean the benchmark stops measuring general capability and starts measuring how hard the lab pushed on that specific distribution.

BIG-bench, a 200-task collaboration explicitly designed to be too broad to game, was an early attempt to push back against this dynamic.5 It worked for a while.

What to do instead

The honest answer is: build your own evals. That is the subject of the next essay in this pillar, Evals vs. vibes.

In the meantime, when you read a launch chart, ask:

  • When was this benchmark published? If it predates the model’s training cutoff by more than a year, assume some contamination.
  • Is the gap larger than the gap between the previous two leaders? Sudden jumps on saturated benchmarks are almost always artifacts.
  • Does the lab show held-out performance on a benchmark they did not pick? If not, treat the chart as marketing.

Footnotes

  1. Hendrycks et al., Measuring Massive Multitask Language Understanding, 2020. ↩

  2. Deng et al., Investigating Data Contamination in Modern Benchmarks for Large Language Models, 2023. ↩

  3. Brown et al., Language Models are Few-Shot Learners, 2020 — Section 4 on contamination. ↩

  4. Rein et al., GPQA: A Graduate-Level Google-Proof Q&A Benchmark, 2023. ↩

  5. Srivastava et al., Beyond the Imitation Game, 2022. ↩

Citations

  1. Measuring Massive Multitask Language Understanding (MMLU) — Hendrycks et al., 2020 · accessed May 17, 2026 · view archived copy
  2. Investigating Data Contamination in Modern Benchmarks for Large Language Models (Deng et al., 2023) · accessed May 17, 2026 · view archived copy
  3. Language Models are Few-Shot Learners (GPT-3) — Brown et al., 2020 · accessed May 17, 2026 · view archived copy
  4. Beyond the Imitation Game (BIG-bench) — Srivastava et al., 2022 · accessed May 17, 2026
  5. GPQA: A Graduate-Level Google-Proof Q&A Benchmark (Rein et al., 2023) · accessed May 17, 2026 · view archived copy

More in Evaluation

  • Evals vs. vibes

    Why every serious AI project ends up writing its own evaluation set, and what a good one looks like.