Detecting Drift in Production AI: Model, Data, and Policy

Detecting Drift in Production AI: Model, Data, and Policy

An AI system that passed evaluation in January can be failing in June without a single commit to your repository. Nothing broke. Nothing threw an exception. Dashboards are green. And yet the outputs a user sees today would not have passed the evaluation you ran six months ago.

This is drift, and most teams monitor for exactly one variety of it — the one their observability vendor happens to chart. In practice there are three, they arise from different causes, they announce themselves through different signals, and they require different remedies. Confusing them wastes the most expensive resource in your evaluation programme: qualified human attention.

Three drifts, not one

Model drift is a change in the system's behaviour when the inputs have not changed. The provider ships a new checkpoint behind the same API name. A safety layer is tuned. A quantised variant is swapped in to reduce serving cost. Your prompt, your retrieval corpus, and your users are identical, but the function mapping input to output is not the one you evaluated.

Data drift is a change in the inputs when the system has not changed. Your users start asking about a product you launched last month. A new customer segment onboards with different phrasing, different languages, different document formats. Your retrieval corpus grows and the nearest-neighbour set for a common query is no longer the set you assumed. The model is behaving exactly as evaluated — on a distribution it was never evaluated against.

Policy drift is a change in what counts as correct when neither the system nor the inputs have changed. A regulator publishes guidance. Legal narrows what the assistant may say about pricing. A clinical guideline is revised. The output that scored 5 on your rubric in March is a finding in June, because the definition of "good" moved underneath it.

The diagram makes the asymmetry visible. Model and data drift change what the system produces. Policy drift changes nothing about the system and everything about the verdict. That is why policy drift is invisible to every automated monitor you own — the monitor is measuring against the old policy.

Detecting model drift

You cannot detect a change in the model by watching aggregate quality scores. Aggregates move slowly and confound all three drifts. You detect model drift with a fixed probe set: a frozen collection of inputs, held constant, replayed on a schedule, with outputs compared against the responses recorded at baseline.

Practical design rules for a probe set:

• Freeze it. The moment you add or edit probes, you lose your comparison baseline. Version it and treat changes as a new baseline, not an amendment.
• Cover behaviour, not just accuracy. Include probes for refusal boundaries, format adherence, citation behaviour, tone, and verbosity. Providers frequently change refusal and verbosity behaviour without changing factual accuracy at all — and refusal changes are the ones that break workflows.
• Pin every controllable variable. Same temperature, same seed where supported, same system prompt, same tool schemas. If you cannot pin sampling, run each probe enough times to distinguish a distribution shift from a sample.
• Alert on distance, not on failure. A probe response that is semantically different but not obviously worse is still a signal. Track output length distribution, refusal rate, citation count, and schema validity as first-class metrics.

Supplement this with the provider's own signals: model version strings in API responses, deprecation notices, changelog subscriptions. These tell you when to look. The probe set tells you what changed.

The uncomfortable case is a silent upgrade that improves general capability while degrading your specific workflow — a model that reasons better but now hedges on the direct answers your users need. Only a probe set anchored to your workflow catches that, because on any public benchmark the new checkpoint looks strictly better.

Detecting data drift

Data drift is the most tractable of the three, because it lives entirely in telemetry you already collect. Watch four families of signal:

Distributional. Embed incoming requests and track the distribution against the evaluation set. Rising distance to the nearest evaluation cluster means you are serving traffic you never assessed. This does not tell you quality has dropped. It tells you your evidence no longer covers what you are doing — which, for a regulated deployment, is the same problem.

Retrieval health. In RAG systems, drift shows first in retrieval. Falling top-k similarity scores, rising rates of queries with no chunk above threshold, and a growing tail of queries whose retrieved context comes from a single document all indicate the corpus no longer answers the questions being asked.

Behavioural. User-side signals are lagging but honest: rising rephrase rates, longer sessions to resolution, more escalations to human agents, more thumbs-down. These do not identify the cause, but they confirm that a distributional shift has become a quality problem.

Structural. New input formats, new file types, new languages, sudden length outliers. These are cheap to monitor and disproportionately predictive, because they mark the arrival of a user population whose needs were never in scope.

Detecting policy drift

There is no metric for this. Policy drift is detected by a person whose job it is to notice, and that only happens if you build the mechanism deliberately.

Three mechanisms work:

Register the sources. Write down every authority that can change the definition of correct for this system: regulators, standards bodies, internal legal, clinical or professional guideline publishers, your own product policy. Assign an owner to each. Review on a fixed cadence.

Version the rubric, not just the model. Your rubric is the operational encoding of policy. When policy changes, the rubric changes, and the change should carry a version and a date. A finding raised under rubric v3 is not comparable to a score awarded under rubric v2, and any report that pretends otherwise is misleading its reader.

Re-score the archive after a policy change. This is the step teams skip. When the rubric changes materially, take a sample of previously passing outputs and re-score them under the new rubric. If they now fail, you have quantified your exposure — and you know whether the change is cosmetic or structural before it reaches a customer.

Turning signals into triggers

Detection is useless without a rule that converts a signal into an action. Define your re-evaluation triggers in advance, in writing, before anyone has an incentive to argue about them.

That last row matters more than it looks. Signal-driven re-evaluation only catches the drift you thought to monitor. A periodic evaluation on a fixed cadence catches the drift you did not anticipate, which is the category that actually hurts you. Treat the calendar as a detector of last resort.

Why drift is an evidence problem

The reason drift belongs in a governance conversation rather than an SRE one is that drift silently invalidates evidence. An evaluation is a statement about a specific system, evaluated on a specific distribution, against a specific definition of correct. Change any of those three and the statement has expired — even though the document still sits in the compliance folder looking authoritative.

This is why structured evaluation records the configuration, not just the score. A SingleAxis Evidence Report is versioned and states what was evaluated, against which rubric, with which taxonomy configuration and severity overrides, by evaluators whose calibration is documented. That makes drift detectable in the evidence itself: when you compare two reports across time, you can see whether the system changed, the data changed, or the standard changed — and you can tell a regulator which.

An unversioned score cannot tell you that. It can only tell you what someone believed on a day you have already forgotten.

What to do tomorrow

Build the probe set — fifty inputs is enough to start, frozen and replayed weekly. Add a version-string check on every provider response. Chart your retrieval no-hit rate. Write down who owns each policy source. Then write your trigger table and get someone senior to sign it, because the value of a trigger is that it fires when it is inconvenient.

Drift is not an anomaly. It is the default state of any AI system exposed to a world that keeps moving. Detection is how you keep your evidence honest.