Building Golden Datasets That Don't Rot

Building Golden Datasets That Don't Rot

Every evaluation you run is a comparison against a set of examples where you believe you know the right answer. That set — the golden dataset — is the ruler. And nobody audits the ruler.

The result is a specific and very common failure: an evaluation programme that has been reporting stable, healthy scores for a year while the golden set beneath it quietly stopped describing the system's actual job. The scores did not lie. They answered a question that no longer mattered.

Golden datasets rot. They rot in at least four distinct ways, on different timescales, and each one requires a different countermeasure.

The four rots

Leakage is the one everyone has heard of and almost nobody controls. Your golden items end up inside the system under evaluation: pasted into a prompt during debugging, added to a few-shot block, absorbed into a fine-tuning run, indexed into the retrieval corpus, or — for public benchmark items — simply present in the model's pretraining data. The system now recognises the question rather than solving it. Scores improve. Capability does not. This is the most dangerous rot because it is invisible in the direction of good news, and good news is not investigated.

Staleness is the mirror image. The item is fine; the label expired. The correct answer to "what is our refund window?" was thirty days when the item was written and is fourteen days now. The system correctly reads the current policy from the current corpus and is marked wrong by a golden set that never got the memo. Teams then "fix" the system to match the stale label, which is how an evaluation programme actively degrades the product it exists to protect.

Scope drift is silent obsolescence. The set was drawn from traffic in a quarter when your users asked about one thing, and they now ask about another. Every item is still valid. Every label is still correct. And your evaluation covers a decreasing share of what the system actually does, so your evidence quietly stops supporting the deployment it is supposed to certify.

Label decay is the uncomfortable one: some of your labels were never right, and nobody noticed because the golden set is treated as ground truth by definition. Or the standard moved — a guideline was revised, legal narrowed what may be said — and every item scored against the old standard now encodes a judgement your organisation no longer holds.

Construction: earn the "golden"

A golden item is not an input-output pair. It is a claim about correctness that a qualified person is prepared to defend, and it needs the structure to support that.

Each item should carry: the input, the expected assessment (an anchored score per rubric dimension, not a free-text ideal answer), the rationale for that assessment, the authority behind it (which policy, which clinical guideline, which regulation, as at which date), the taxonomy codes it exercises, a difficulty band, and a provenance record — where the item came from, who labelled it, who reviewed it, when.

Three construction principles do most of the work.

Label the assessment, not the answer. For generative systems there is no single correct output, so "expected output" is a category error that produces brittle string comparisons. What you can specify is how a correct output must score: which dimensions must clear which anchors, which taxonomy codes must not fire. This is why gold tasks live against the rubric rather than beside it — the golden item and the production item are scored through the same instrument, which is the only way the comparison means anything.

Double-label everything, from the start. An item labelled by one person is that person's opinion with a promotion. Golden items must be independently assessed by at least two qualified Evaluators and reconciled through adjudication, with the adjudication rationale retained. If two qualified people cannot agree on the correct assessment of a candidate item, it is not golden. It may be something better — a boundary case that shows your rubric is underspecified — but it cannot be a ruler.

Cover the hard cases deliberately. A golden set drawn purely at random from production traffic will be dominated by easy items, because production traffic is. It will produce a high, stable, uninformative score. Stratify: routine cases to detect regression, boundary cases to locate the standard, adversarial cases to probe robustness, and unanswerable cases to test refusal. Record the stratum on the item, and report per stratum. A blended score across strata is an average of quantities that are not comparable.

Anti-leakage discipline

Leakage is a containment problem, not a data-cleanliness problem, and it needs mechanical controls rather than good intentions.

• Hold back a sealed slice. A portion of the golden set is never shown to the building team, never used for prompt iteration, never used to select a model. It exists solely to answer the question "does the score we get on the visible set generalise?" When visible-set performance and held-back performance diverge, you have measured your leakage directly.
• Never let golden items enter any system asset. Not the retrieval corpus, not the few-shot examples, not a fine-tuning set, not an evaluation prompt shown to a vendor. Enforce this with a hash check in CI against corpus and prompt assets, because it will otherwise happen by accident within a quarter.
• Rotate rather than accumulate. Retire a share of items each cycle and replace them with fresh ones drawn from recent traffic. An item that has been in circulation for two years has had two years of opportunity to leak.
• Prefer private items for anything consequential. Public benchmark items are, by construction, likely to be in pretraining data. They are fine for orientation. They cannot support a claim about your system.
• Treat an unexplained jump as a leak until proven otherwise. A sudden improvement with no corresponding change to the system is not a win. It is an investigation.

Refresh cadence

The row that gets skipped is the rubric-revision row. When the standard changes, the golden set is not automatically still golden — it was labelled against the previous definition of correct. Re-scoring the set under the new rubric is how you find out whether the change was cosmetic or structural, and the items that flip from pass to fail are the most informative artefact the revision will produce.

Golden items as a live calibration instrument

The highest-value use of a golden item is not batch scoring. It is injection: dropping golden items unannounced into the live evaluation queue, indistinguishable from ordinary work.

That single mechanism yields three things a batch run cannot. A per-Evaluator detection rate, measured continuously rather than at onboarding. An early warning when someone's interpretation of a dimension drifts mid-engagement. And an honest answer to the question an auditor will eventually ask: how do you know your Evaluators were applying the standard correctly on the day they scored this?

The answer "they were trained" is an assertion. The answer "here is their measured accuracy on gold tasks injected throughout the engagement, and here is the adjudication record for the cases they missed" is evidence — and it is what lets the verdicts in an Evidence Report carry weight beyond the reputation of the person who signed them.

What to do tomorrow

Pull twenty items from your golden set at random and ask a domain expert to re-verify the label from first principles, without seeing the existing one. Count how many they change. That count is your label decay rate, and it is almost never zero.

Then hash every golden input and grep it against your retrieval corpus, your prompt templates, and your fine-tuning data. Finding a match is not embarrassing — it is the first honest measurement your evaluation programme has produced in a while.

Then seal a held-back slice, write down your rotation cadence, and put a date on every label. A ruler with no date on it is not a ruler. It is a souvenir.