Article URL: https://elman.ai/news/your-model-already-knows-the-answer/ Comments URL: https://news.ycombinator.com/item?id=49185536 Points: 6 # Comments: 0

The best test of an AI model is a real world problem with a known answer. It is also the easiest test to cheat. An AI model has read much of the internet, so it may have met your question, the answer or both long before you asked. A high score then hides two very different things: a model that reasoned its way to the answer, and one that is repeating something it already read. Telling those apart is the contamination problem, and it sits under a surprising share of the benchmarks the field relies on. Imagine you want to know how reliably a model can predict whether a drug will succeed in the clinic. Rather than wait years for fresh trials, you test on ones that have already finished: take drug programmes that reached late-stage trials, hide whether each worked, and ask the model to predict. On paper it looks like a clean test, fast and scored on real outcomes, but it is not: the trials worth testing are often the famous ones, and their outcomes recur across the textbooks, reviews, news and patents the model trained on. When the model says a drug will succeed, the score cannot separate a model that reasoned its way there from one that recognised the name and recalled what happened. Clinical trials are our field, but the trap is general: it springs on any benchmark built from resolved events, in finance, law, forecasting or anywhere the outcome is already public. These benchmarks are built from public history for a good reason: a real outcome is a better test case than any expert can invent, because it reflects true biology and the real ways a drug fails, not just what a question-writer thought to ask. Those outcomes are the gold standard we want to score against, and being public is exactly what makes them leak. So, what can we do? The answer can reach the model by three routes. This split is our own way of organising the problem rather than a named taxonomy, though each route has an established basis. Input leak happens at test time, through the documents the model reads for the case in front of it, whether you feed those documents in or the model fetches them itself with a search tool. Any document written after the decision date can announce the very outcome the model is meant to predict, which is as good as letting it read the future (Kaufman et al., 2012). Benchmark leak happens when a model is released after your benchmark and the model might have been trained on the benchmark set itself, memorising both the questions and answers. This is the most common way in which a benchmark decays over time. Balloccu and colleagues traced more than four million test samples leaking into GPT-3.5 and GPT-4 within a year of their release (Balloccu et al., 2024), and when other researchers rewrote a set of grade-school maths problems from scratch, several model families scored markedly lower on the fresh version (GSM1k, Zhang et al., 2024). Outcome leak happens when the outcome is itself a public fact and the model has trained on it: a completed trial, a regulatory approval or a drug that worked is repeated across thousands of papers and filings, and the model absorbs it during training, whether in pre-training or in the later fine-tuning that follows, with no contact of any kind with your particular benchmark. The model comes to hold the result in the same way that it holds any other widely reported fact, such as the year in which a war ended. As a consequence, you can write a completely fresh test set that the model has never seen and date-blind every input you give it, and the model will still know that this well-known drug succeeded. Keeping the benchmark secret does not help either, because the answer was never contained in your file in the first place; it was already part of the model’s general knowledge of the world. The cleanest way to prevent any type of leak is to stop testing on history altogether and to score the model only on events that have not yet happened, so that there is simply no answer available to leak. Live benchmarks refresh their questions on a fixed schedule (LiveBench, White et al., 2024), and forecasting benchmarks like ForecastBench score models on events that have not yet resolved at the time of the test (ForecastBench, Karger et al., 2024). CT Open is a live platform that scores models on clinical trials before their results are public: predictions go in before each challenge opens, and an automated pipeline, validated against expert annotation, searches out the earliest public mention of every outcome, so a trial counts only if its answer was nowhere to be found when the prediction was made (Wang et al., 2026; ct-open.net). Trial results land all year round, and CT Open scores four times a year to match. The decision we care about is a slower one: the commitment that costs a billion dollars is taken about a decade before the readout that settles it, so put that decision to a live benchmark and your first score arrives in 2036.