The Benchmark Collapse

The instruments stopped measuring

Every scientific field runs on instruments, and instruments fail in two ways. They can break loudly, which is inconvenient, or they can keep producing numbers after they have stopped measuring anything, which is worse. LLM evaluation is drifting into the second condition. The leaderboards still update. The technical reports still tabulate. And the numbers, increasingly, cannot support the conclusions being drawn from them.

Before the evidence, the claim — stated precisely, because the word "collapse" invites misreading. This essay does not argue that benchmarks have no diagnostic value. It argues that public static scores can no longer bear the institutional weight placed on them as general rankings of model capability. A saturated or partially contaminated benchmark can still track progress within a fixed model family, still discriminate among weaker models, still serve as a regression test. One industry evaluation guide puts the residual value plainly for the most famous example: "MMLU scores no longer differentiate between leading models. It remains useful for evaluating smaller and mid-tier models where the spread is still informative" [4]. What such a benchmark can no longer reliably do is the one thing everyone still uses it for — answer the public question which frontier model is better, and by how much? That specific function, the general-ranking function, is what has collapsed.

Start with the plainest symptom: saturation. A systematic study of 60 text benchmarks drawn from the technical reports of major model developers — a multi-institution preprint with companion code, not yet peer-reviewed — found that 29 exhibit high or very high saturation, 14 of them in the top category, where saturation is defined as "the loss of reliable discriminative power among state-of-the-art models" [1]. HumanEval, once the standard coding evaluation, shows an average score of 0.8 across 66 evaluated models on one public tracker, against a test set of only 164 problems; when Codex introduced the benchmark it solved 28.8% of those problems and GPT-3 solved 0% [2]. The organizers of Humanity's Last Exam justified building a new instrument in exactly these terms: models now exceed 90% accuracy on popular benchmarks like MMLU, which forecloses informed measurement of the frontier [3]. By early 2026, commentary aimed at practitioners was treating MMLU, GSM8K, and HumanEval as saturated for frontier comparison purposes as a matter of course [4].

The comfortable reading is that saturation is a success story — the tests got passed because the students got smart. That reading is available, and I will return to it at full strength, because it deserves a real answer rather than a wave. But the saturation study itself points somewhere less comfortable. It found that saturation is associated primarily with structural exposure dynamics and with measurement resolution limits such as test set size, not with isolated design choices — and, strikingly, that public benchmarks did not saturate detectably faster than private ones in its sample [1]. If that finding holds up, the instruments are not failing because someone made a bad design decision that a better committee could have avoided. They are failing because of what benchmarks structurally are: small, fixed, widely exposed objects embedded in a system that consumes them. That is the collapse thesis in one sentence. What broke is not any particular test. What broke is the assumption that a static public test can remain an instrument once an entire industry is optimized against it.

Seven ways a benchmark fails

Before going further, the argument needs a distinction the debate usually skips. The benchmarks are broken bundles together at least seven different failure modes, and they are not the same thing. Saturation can come from genuine capability gains. Contamination does not always produce measurable inflation. Overfitting can occur without verbatim contamination — the GSM1k authors are explicit that contamination is likely not the full explanation for the gaps they measured [8]. Construct invalidity survives perfect test secrecy: a private exam can still be the wrong exam. And a leaderboard's governance can fail while every question on it remains fresh. Conflating these modes weakens both the criticism and the defense, because a rebuttal to one mode gets misread as a rebuttal to all of them.

The table below separates them. Each cell is backed by the cited evidence; where a source is a preprint, vendor report, or commentary rather than a peer-reviewed study, the surrounding sections say so.

Two consequences follow from this separation. First, the fix depends on the mode. Only saturation is even temporarily addressed by writing harder questions; every other row survives a harder test set untouched, and the last six months supplied a live demonstration of how quickly the temporary fix erodes, which a later section takes up. Second, the modes compound. A benchmark can be simultaneously saturated, contaminated, and construct-invalid — the evidence below suggests several are — and a defense that addresses one mode while the others operate is not a defense of the score.

Contamination is the system working as designed

The word "contamination" suggests an accident — something leaked where it should not have. The evidence suggests something closer to metabolism. Frontier models are trained on web-scale corpora that scrape the same internet where benchmark questions, answer keys, and discussion threads live. A survey of contamination research published in the Findings of ACL — peer-reviewed — documents how test data enters training through arXiv papers, documentation sites, and Q&A forums, and notes that OpenAI used n-gram string matching to detect such collisions for GPT-3 [5]; a practitioner reference on detection methods describes the standard threshold at which overlap becomes damning, since matching 13 consecutive tokens is vanishingly unlikely by accident [14]. The pipeline that makes models capable is the pipeline that dissolves their exams.

The field's own safeguards tell the story most clearly. BIG-Bench embedded a canary string — a unique GUID whose entire purpose is to let labs filter benchmark files out of training data. An independent investigation found that GPT-4-base had memorized tasks containing that canary, perfectly recalling completions for 19 tested tasks [6]. Documented fact and editorial inference should be kept apart here. The documented facts: this is a single investigator's post, not a peer-reviewed study, but the underlying disclosure is corroborated by the GPT-4 report itself, which excluded BIG-Bench as an evaluation because of training-data contamination [6]. The inference, which is mine: when the mechanism explicitly designed to keep test data out of training corpora fails at the most-resourced labs in the field, contamination stops being a hygiene lapse and becomes a built-in property of the training pipeline.

Its measured effects are not small. A contamination-threshold study across 13 benchmarks and 7 models — reported here through a vendor's research write-up, so the specifics warrant the hedge — suggests that contamination inflates scores more than previously reported, with Llama 1 gaining over 20% on several benchmarks from contaminated examples, and that larger models exploit contamination more effectively than smaller ones [7]. That last finding should give leaderboard readers pause: the models most likely to top a chart are the models best equipped to profit from a compromised one.

The cleanest experiment in this literature is GSM1k. Researchers commissioned 1250 grade-school math problems, written entirely by human annotators and calibrated to mirror GSM8K in difficulty and style — a genuinely controlled comparison — then re-evaluated the field [8]. The annotators were sourced by Scale AI, which also sells private evaluation services and later built its SEAL math leaderboard on GSM1k [19, 20] — an interest disclosed here for the same reason Cohere's is disclosed below. In the peer-reviewed NeurIPS 2024 version, models' accuracy dropped by up to 8% relative to their GSM8K scores, with several families, notably Phi and Mistral, showing systematic overfitting across almost all model sizes; the earlier preprint had reported drops of up to 13%, and the shrinkage between versions is itself a lesson in why headline figures deserve suspicion [8, 9]. The peer-reviewed version also found a positive relationship between a model's probability of generating verbatim GSM8K test examples and the size of its performance gap [9] — memorization, measured directly. (The paper reports this as Spearman's r² = 0.36; whether that denotes the coefficient or its square is ambiguous in the text, so I report it as the paper's statistic, not as a correlation coefficient.) A model's published math score, in other words, was partly a measurement of its math and partly a measurement of how much of the answer key it had eaten. A number that mixes those two quantities in unknown proportion is not a measurement. It is a rumor with decimal places.

One more development moves this from history to present tense. Contamination detection has been an arms race between detectors and training pipelines, and the race just entered a phase where the standard detectors go blind. A paper accepted at ICLR 2026 shows that contamination introduced during reinforcement-learning post-training — now a core stage of frontier model development — evades likelihood-based detectors almost entirely: on the authors' purpose-built benchmark, standard methods performed close to random guessing (AUC around 0.50), and their proposed entropy-based probe, while better, reached only about 0.70 [10]. Even where labs sincerely audit for contamination, the auditing tools were built for a training paradigm the field has partly moved past.

Goodhart, with asymmetric access

If contamination corrupts the written exam, the field's proposed escape was the live one: Chatbot Arena, where anonymous head-to-head human votes would measure what no training corpus could contain. The record of that experiment is now the sharpest evidence for the institutional version of the collapse thesis, because the Arena failed in a way no test-hardening could have prevented — through its incentive architecture.

"The Leaderboard Illusion," a preprint from researchers at Cohere Labs, AI2, Princeton, Stanford, Waterloo, and the University of Washington — note that Cohere competes on the leaderboard it is criticizing, an interest that cuts both ways — documented that undisclosed private testing let a handful of providers evaluate many variants before release and publish only the winner. "At an extreme, we identify 27 private LLM variants tested by Meta in the lead-up to the Llama-4 release" [11]. Of 243 public models on the platform, 205 had been silently deprecated, with open-weight models sampled less and removed more [11]. The same paper estimated that modest increases in access to Arena data could boost a model's Arena performance by up to 112% [12] — the Arena's own interaction logs became training fuel, so being sampled often was itself a compounding advantage. The Llama 4 launch supplied the reductio: the model that topped the leaderboard was not the model released to the public [11].

Here the documented record and my inference from it need explicit separation. The documented facts establish asymmetric access: private variant testing existed, disclosure was optional, score retraction was permitted, and sampling rates differed by provider [11, 12]. My inference is the general law: any evaluation that is simultaneously public, high-stakes, and permeable to iterated private probing will converge to the same state, because it systematically rewards exactly the actors with the most riding on the result. Goodhart's law describes a measure degrading once it becomes a target. What the Arena record adds is the differential version — a target that a few well-resourced actors could probe repeatedly, privately, and retractably, while everyone else submitted once and lived with the result. Commentary treated the paper as a scandal about one leaderboard. The systemic reading is less comforting and more useful: the Arena was not an aberration among public evaluations; it was the best-instrumented one, which is why we caught it.

Fairness demands the other side of the record. LMArena's co-founder disputed the study's framing, and a Google DeepMind researcher stated that Google submitted only one model variant for pre-release testing [12]. The dispute is real, and one of the paper's senior authors framed the stakes in words both sides could sign: "It is critical for scientific integrity that we trust our measure of progress" [12]. But note what is not disputed — that private variant testing existed, that disclosure was optional, and that the platform's policies permitted retraction. A measurement institution whose integrity depends on the voluntary restraint of the parties being measured has already failed, whether or not every party exploited it.

The strongest counterargument, taken seriously

The defense of benchmarks comes in three forms, and the honest version of this essay has to meet all of them at full strength.

The first says benchmarks still predict capability — that for all the noise, scores correlate with genuine quality. There is evidence for this. LiveBench's rankings correlate at 0.88–0.91 with Chatbot Arena and Arena-Hard [15]. The GSM1k study itself found that frontier models showed minimal signs of overfitting and that all evaluated models demonstrated real generalization to problems guaranteed absent from training [9]. And intellectual honesty requires a further concession at the point where it costs the most: the collapse literature is itself uneven. The GSM1k headline figure shrank from 13% to 8% between preprint and peer review [8, 9]; the Arena findings are contested by named parties [12]; the saturation study is not yet peer-reviewed [1]; and several sources in this debate are vendor blogs with products to sell, flagged as such throughout. If the thesis required every score to be fraudulent, it would fail.

But the thesis requires no such thing, and this is where the predictive defense misses its target. That instruments correlate with each other does not establish that either measures the construct — two thermometers in the same warm room agree, and their agreement tells you about the room, not about either instrument's calibration. The deeper problem is validity, and here the peer-reviewed record is blunt. An ICML 2025 oral position paper showed that medical benchmarks built from licensing-exam questions fail to capture real clinical tasks, with top-scoring models performing poorly against real hospital data, and argued that the field must import construct validity — "the ability of a test to measure an underlying construct, that is the actual conceptual target of evaluation" — as a precondition for using any benchmark at all [16]. Outside medicine the same gap recurs: a study of class-level code generation on real repositories, using post-cutoff classes guaranteed unseen by the models, reports a stark performance disparity between synthetic coding benchmarks and authentic software projects [17]. The predictive defense is an empirical claim, and where it has been tested against ground truth rather than against other benchmarks, it keeps losing.

The second defense says saturation is progress — the tests got passed. Sometimes, surely; models did get better at math, code, and knowledge retrieval by any measure. But a passed test and a dissolved test produce the same number, and the burden of the contamination record is that we cannot tell which we are looking at. GSM8K scores near ceiling were read as arithmetic mastery until GSM1k showed that for several model families a measurable fraction of the ceiling was memorization [8, 9]. Saturation would be progress if scores were readable. Contamination makes them unreadable. An unreadable instrument at ceiling is not a trophy; it is a dead gauge pinned at maximum.

The third defense is the newest and the strongest: the ecosystem is self-correcting. Every damning study in this essay — GSM1k, the Leaderboard Illusion, the canary investigation, the audits described below — was produced by the field examining itself, which suggests the institution works. I concede the premise and contest the conclusion. Each correction was achieved by stepping outside the public static benchmark: commissioning fresh human-written problems, analyzing platform logs, probing base models directly, auditing test suites by hand. The self-correction is real, and it is precisely the audit-shaped activity this essay ends by endorsing. What self-corrected was the field. What did not self-correct — what had to be corrected from outside, every time — was the public score.

What changed in the last six months

A thesis about institutional failure should make loose predictions, and the period from roughly late January to July 2026 offers a test: if the collapse is structural, we should see the evaluation system visibly reorganizing around it — retreating from public static scores, building private and dynamic instruments, and formalizing audits. That is a fair description of what the record shows.

The most consequential event came in February 2026, when OpenAI stopped evaluating against SWE-bench Verified — a benchmark it had itself curated — after an internal audit (secondary sources disagree on whether the cessation dates to September 2025 or February 2026; the audit details are consistently dated February 2026); secondary reporting states the audit examined 138 tasks its model repeatedly failed and found 59.4% of them flawed, with tests that rejected correct solutions, and evidence that frontier models had seen benchmark tasks during training [24, 25]. Scores reported around 70% on Verified reportedly fell to around 23% on the harder SWE-bench Pro public split [25]. A 2026 preprint on coding-benchmark misalignment records the resulting position: SWE-bench Verified, in OpenAI's words, "no longer measures frontier coding capabilities" [26]. The interest structure here deserves naming: OpenAI is simultaneously the auditor, a competitor scored by the benchmark, and the promoter of successor evaluations including privately authored ones [25], so the episode is best read as a claim by an interested party whose direction of travel — away from the public static score — is more informative than its exact numbers. One practitioner write-up drew the general moral in a sentence worth keeping: "When the majority of your benchmark penalizes right answers, you're measuring which models accidentally align with broken assumptions, not actual capability" [24].

Private internal evaluation, meanwhile, moved from hedge to doctrine. Microsoft AI's technical report for its MAI-Thinking-1 model — a vendor primary source describing its own methodology — describes a suite of nearly 40 internal benchmarks built to be unmeasurable by the public internet: "vendors who have created data fully held out from training, internal sources not present on the web, and web sources that we are careful to remove from our training data" [23]. Whatever else this is, it is a frontier lab acting on the premise of this essay: that a score is only meaningful if the test provably never touched the training data, and that the only way to guarantee that today is to keep the test off the internet entirely.

The dynamic and interactive wing advanced too. The ARC Prize Foundation launched ARC-AGI-3 in the spring of 2026 — the launch event was held March 25 — as a fully interactive benchmark of hand-built game-like environments with no stated rules or goals, scored partly on semi-private and fully private evaluation sets. Its launch numbers are the starkest headroom statement in current evaluation: "Humans score 100%. Frontier AI scores 0.51%" [27, 28]. The design is a direct answer to three failure modes at once: private sets resist contamination, interaction resists memorization, and skill-acquisition scoring targets a construct — learning efficiency — rather than recall [28]. One caveat belongs on the record: a single commentator's account, uncorroborated elsewhere in this evidence, describes an unannounced difficulty change to one environment days before launch and a human baseline established over days rather than months — a reminder that even the newest instruments launch with governance wobble.

But the record also demonstrates — once within the window, once from 2024 — why none of this is a resolution. First, harder questions decayed on schedule. Humanity's Last Exam launched with frontier accuracy far below saturation — its own page listed Gemini 3 Pro at 38.3% and GPT-5 at 25.3% at the time of capture [3]. By July 2026, third-party trackers suggest the ceiling has risen fast — one reports a top score of 40.0% while another lists a figure more than twenty points higher — and the fact that two trackers disagree by more than twenty points on the same benchmark — one of them apparently months stale — is itself a finding about the state of public measurement infrastructure [29, 30]. The HLE organizers predicted this decay from the start, writing that models could plausibly exceed 50% accuracy by end of 2025 [3]. Harder data buys resolution; it does not buy trust, and it depreciates.

Second, private evaluation showed its own characteristic failure, and it is governance, not exposure. Epoch AI's FrontierMath, a benchmark of unpublished expert math problems built to be contamination-proof, turned out to be funded by OpenAI, which had commissioned and owned 300 of its questions with access to statements and solutions, facts not disclosed until December 20, 2024; Epoch withheld a randomly chosen set of 50 solutions as a holdout, and its co-founder conceded: "we should have made transparency with our contributors a non-negotiable part of our agreement with OpenAI" [33, 34]. A private benchmark relocates trust from the test items to the assessor — and the assessor, it turned out, had an undisclosed relationship with the party being assessed.

The audit layer, finally, began to institutionalize. OpenAI published its framework for external testing, in which third-party labs run open-ended assessments on pre-release checkpoints: "Third party assessors add an independent layer of evaluation alongside our internal work, strengthening rigor and providing additional protections against self-confirmation" [31] — though the same document shows assessors sign NDAs and submit publications for approval, which bounds the independence on offer. A secondary compilation on third-party auditing records the surrounding scaffolding: formal testing agreements between the US AI Safety Institute and frontier labs signed in August 2024, METR's pre-release evaluations of frontier models, and the EU AI Act's third-party conformity assessment regime reaching full applicability in August 2026 [32]. Judge reliability got its own audit tooling: reporting on recent judge-evaluation research describes an open-source stress-testing harness released by RAND researchers in March 2026, with the finding that "No judge evaluated by the RAND Corporation's research team was uniformly reliable across benchmarks. Frontier models exceeded 50% error rates on bias tests" [22] — consistent with the peer-reviewed position-bias literature showing systematic, non-random position effects across 15 judges [21]. Evaluation-by-rubric, the scoring method of last resort for open-ended tasks, is itself an unstable instrument, and the field now builds instruments to measure its instruments.

Six months, then, produced: a lab abandoning its own public benchmark, a lab building 40 tests the internet cannot see, an interactive benchmark with private splits, a hard public exam depreciating on schedule, the unresolved lesson of a 2024 private-benchmark disclosure failure, and audit agreements hardening into law. None of this is what a healthy public-scoreboard institution looks like. All of it is what a measurement system under reconstruction looks like.

What repair looks like

If the collapse is structural, the repair must be structural — not better questions but different institutional forms. Three are legible in the evidence, and they matter because each removes a failure mechanism rather than racing against it. None survives as a solution; each survives as a mechanism with a named residual risk.

The first is the private held-out evaluation, which removes exposure. Scale AI's SEAL leaderboards — described here from the vendor's own launch materials, and built by the same vendor that produced GSM1k — keep proprietary datasets of roughly 1,000 examples each unpublished, score models with vetted human experts, and restrict repeat probing [19, 20]; Microsoft's internal suite applies the same logic at the development stage [23]. The logic is the auditor's: you cannot study for an exam you have never seen. The residual risk is now documented rather than hypothetical. Private evaluation converts contamination risk into assessor-trust risk, and the FrontierMath disclosure failure shows assessor-trust risk realizing in practice — an undisclosed funder that owned the questions [33, 34]. Institutional credibility is at least a problem institutions know how to manage; but it is managed with disclosure rules and independence guarantees, not with dataset secrecy alone.

The second is the dynamic or interactive benchmark, which removes staleness. LiveBench regenerates questions monthly from fresh competitions and current sources and scores against objective ground truth without an LLM judge — a design choice vindicated by the observed LLM-judging error rates of up to 46% on hard problems — and at introduction no model exceeded 65% accuracy [15]. Humanity's Last Exam, assembled from nearly 1,000 expert contributors across over 500 institutions in 50 countries, pushed difficulty to the expert frontier [3]; ARC-AGI-3 pushes past static questions entirely, into environments that must be explored rather than answered [27, 28]. Headroom is the precondition for measurement, and these instruments have it — on a clock. The HLE trajectory shows the depreciation schedule for hard static questions is now measured in months, not years [3, 29, 30]; refresh cycles and interactivity slow the clock, and nothing in the evidence stops it.

The third is the task-based evaluation, which removes the proxy. SWE-bench Verified poses 500 real GitHub issues scored by whether the repository's own tests pass; GAIA poses 466 multi-step web and file tasks; OSWorld poses 369 desktop tasks checked by execution [35]. These are not quiz scores we hope transfer to work; they are the work, sampled. The residual risks are now well documented. The reference tests themselves can be wrong — the audit finding, per secondary reporting, that 59.4% of examined Verified tasks had flawed tests [24, 25]. The harness confounds the measurement: an analysis found a model resolving 31% of sampled Verified issues agentically solved 0% of them in a single long-context pass [36], and a 2026 preprint reports the same model scoring anywhere from 58.0% to 79.8% on Terminal-Bench depending on which agent harness wraps it [26]. A task-based score has a floor of meaning a saturated multiple-choice percentage lacks — the tests passed on real code — but what it measures is a system, never a model alone.

Beneath all three runs the reframing that the peer-reviewed literature keeps converging on: evaluation as audit rather than tournament. A journal article in Patterns, writing in the context of AI fairness evaluation, argues for replacing single-score leaderboards with curated benchmark suites that surface trade-offs instead of crowning winners [18]; the construct-validity position paper demands that an instrument earn the right to judge models by first being judged against reality [16]. The common thread is the death of the single number — the scalar that ranks all systems for all purposes — and its replacement by an inspection regime: private probes, refreshed items, executed tasks, published methodology, adversarial review, disclosed interests.

What this means for labs, buyers, and evaluators

For labs, the actionable failure in the record is not contamination — which the evidence suggests no web-scale pipeline fully prevents [5, 6] — but nondisclosure. The events that became scandals share one shape: private variant testing without disclosure [11], funder relationships without disclosure [33], benchmark abandonment announced after the scores had served their purpose [24]. The implication runs in the lab's own interest: a lab that publishes its evaluation access conditions — how many variants were tested, what was retracted, who funded the instrument, what the canary-compliance process is — is buying credibility at exactly the moment public numbers stop carrying it. Internal held-out suites of the kind Microsoft describes [23] solve the lab's measurement problem; only disclosure solves its trust problem.

For buyers, the practical rule follows from construct validity: a public score is a screening filter, never procurement evidence. The medical result generalizes — models that top exam-style benchmarks underperformed on real clinical data [16], synthetic coding scores did not transfer to real repositories [17], and even a platform-friendly guide to the Arena advises using it as "a starting filter, not a final procurement decision" [13]. The deployment-facing versions of the same advice converge from every corner of the evidence: evaluate on your own tasks, your own repositories, your own workflows, because that evaluation is the only one whose construct is, by construction, the thing you care about [17, 24]. Where a vendor cites a score, percentages at a stated date carry information; ranks carry almost none, since rank shifts with the voting pool, the harness, and the deprecation policy [11, 26].

For evaluators — benchmark authors, leaderboard operators, auditors — the record reads as a design checklist. Validate the construct against ground truth before publishing a leaderboard on it [16]. Keep a genuinely held-out split, and disclose who has access to the rest, under what funding [33, 34]. Refresh on a schedule faster than the depreciation you observe [15, 3]. Score by execution where possible; where a judge model is unavoidable, stress-test it, because the judge literature shows reliability failures that are systematic rather than random [21, 22]. Publish deprecation and retraction policies before the first score, not after the first controversy [11]. And report what the harness contributes, because a leaderboard that cannot distinguish the model from its scaffolding is ranking engineering teams while claiming to rank models [26, 36].

The benchmark era gave the field something real: a shared direction, a common language, a decade of measurable ascent. What it can no longer give is the thing everyone still uses it for — a trustworthy public answer to the question which model is better? The institutions that will answer that question next are being built now, in private eval suites and audit agreements and execution harnesses, partly out of public view. Which means the practical question facing anyone who deploys, buys, regulates, or reports on these systems is no longer what did it score? It is the auditor's question, and it has always been the better one: who measured this, with what access, against what reality — and what would they have seen if it were false?