Regression Testing for AI: Gates That Actually Block a Release
Regression Testing for AI: Gates That Actually Block a Release
Software engineering solved this thirty years ago. You change the code, the test suite runs, a test fails, the build is red, the change does not ship. The gate is binary, it is trusted, and nobody argues with it — because a failing assertion is not a matter of opinion.
AI teams have tried to import this and produced something that looks similar and behaves nothing like it. The pipeline runs an evaluation. The evaluation produces a score. The score is written to a dashboard. The build goes green. Somebody looks at the dashboard on Thursday, if they remember.
That is not a gate. A gate stops things.
The reason AI teams end up here is not laziness. It is that a genuine gate requires answering a question the team has been avoiding: what result is bad enough to block a release? Nobody wants to be the person whose threshold blocked a launch, so the threshold is never set, and the evaluation degrades into telemetry.
The four things that make a gate real
It is deterministic enough to be trusted. If the same commit produces a passing run and a failing run, engineers will learn to re-run until green, and your gate is dead. Pin temperature to zero where the workload permits, fix seeds where supported, pin the provider model version explicitly, and — where sampling cannot be removed — run each case enough times to test a rate rather than a single draw, then gate on the rate. A flaky gate is worse than no gate, because it teaches people to ignore red.
It has a threshold that was set in advance. Before the run. Written down. Owned by someone with the authority to hold it. A threshold negotiated after seeing the result is not a threshold; it is a rationalisation.
It fails the build. Not a warning. Not an annotation. The pipeline stops.
It has an override path that costs something. Absolutism does not survive contact with a launch date. There must be a way through — but it must require a named human, a written justification, and a record. If overriding is a click, everything is overridden. If overriding requires a director to type a reason that lands in the release record, it happens only when it should.
Three tiers, not one
A single blended "eval score" gated on a threshold is the most common design and the worst one, because it lets a catastrophic safety failure be averaged away by improvements in tone. Separate the suite into tiers that fail for different reasons.
Tier 1 — assertions. Binary, deterministic, non-negotiable. Does the output parse as valid JSON against the schema? Does it ever emit a national insurance number from the retrieval context? Does it refuse the twelve prompts it must always refuse? Does it call the tool with the right signature? Does it still get right the forty cases from previous incidents?
These are ordinary unit tests that happen to have a model in the loop, and they should be treated with ordinary unit-test seriousness: any failure is red. Most teams have far fewer Tier 1 assertions than they could, because they assume everything about an AI system is fuzzy. A great deal of it is not.
Tier 2 — the scenario suite. Scored, not asserted. A fixed, versioned set of realistic cases, evaluated against the rubric, gated on aggregate quality. This is where judgement lives, and it needs two design decisions most teams get wrong.
First: gate on the lower bound of the confidence interval, not the point estimate. A scenario suite is a sample. A mean score of 4.1 against a floor of 4.0 is not a pass if the interval runs from 3.8 to 4.4 — you are gating on noise. Compute the interval and gate on its lower bound, and if the interval is too wide to make a decision, your suite is too small.
Second: gate on the per-dimension floor as well as the aggregate. Otherwise a model that got noticeably worse at factual grounding and noticeably better at conciseness sails through with an unchanged average. Aggregate scores hide exactly the trade you most need to see.
Tier 3 — exploration. Report-only by design: open-ended probes, adversarial prompts, novel scenarios, drift checks against production distribution. It does not gate, because a gate on an unstable, evolving suite would be flaky and would be ignored. Its job is to manufacture Tier 1 assertions and Tier 2 cases. That feedback arrow in the diagram is the mechanism by which the suite gets better over time, and a programme without it is testing for last year's failures forever.
Silent regressions: what a suite never catches
The most dangerous regressions are the ones your suite was not built to see, and they have a characteristic shape.
The trade-off regression. The model improves on your suite's dominant category and degrades on a minority one. Aggregate up, real capability down for a population you underweighted. This is why per-dimension and per-stratum floors are not a refinement but a requirement.
The overfit-to-suite regression. Your scenario suite has been in use for eighteen months. Prompts have been tuned against it. Retrieval has been tuned against it. The suite is now a training signal, and its score has stopped predicting anything about production. Every fixed evaluation set decays this way. The countermeasure is a held-out suite — a portion of cases that the team building the system never sees, refreshed periodically from production, run only at gate time. If your gated score is excellent and your held-out score is mediocre, you have measured the overfit exactly.
The behavioural regression. Accuracy is unchanged; something else moved. The model now hedges where it used to answer. Refusals rose 6%. Average output length doubled, which broke a downstream parser. Citation count fell. None of these show up in a correctness metric, all of them break workflows, and all of them are cheap to assert on. Track output length distribution, refusal rate, and schema validity as first-class gated signals.
The tail regression. Aggregate flat; the worst 1% got much worse. If your gate looks only at the mean, a model that is slightly better on average and catastrophically worse on the hardest cases passes. Gate on a tail statistic too — the count of cases scoring below the finding threshold must not increase.
Who owns the verdict
There is a governance question buried in the mechanics, and it becomes acute the moment a gate blocks a launch.
If the team shipping the model also owns the suite, the thresholds, and the override, the gate is a formality with extra steps. It will be adjusted, quietly and reasonably, in the direction of shipping. Nobody will be acting in bad faith; the incentives will simply do their work.
The separation that makes gates credible is the same one that makes financial controls credible: the evidence is produced by people who are not measured on the release date. For consequential systems, that means the rubric is defined independently of the delivery team, the scoring is performed by credentialed Evaluators, the evaluators are themselves calibrated with gold tasks, disagreements between them are adjudicated rather than averaged, and the result is recorded in a versioned Evidence Report that says what was evaluated, against which rubric version, with what verdict.
The pipeline gate then reduces to something a machine can enforce: does a current, passing evaluation record exist for this exact configuration? If the configuration changed, the record does not apply, and the gate is red — not because anyone judged the change to be bad, but because nobody has yet judged it at all.
That is the property you actually want. The default state of an unevaluated change is blocked.
What to do tomorrow
Take your most recent AI-related production incident. Write a Tier 1 assertion that would have caught it, and put it in the pipeline today. That single test is worth more than a dashboard, because it can fail.
Then do the arithmetic on your scenario suite: compute the confidence interval on its aggregate score. If the interval is wider than the difference you are trying to detect between two model versions, your gate has never actually been measuring anything — and now you know how many more cases you need.
Finally, ask who is allowed to override the gate, and check whether it has ever been used. A gate that has never blocked anything is either a system that has never regressed, or a gate that does not work. It is not usually the first.