Chain of Custody for AI Evaluation Evidence

Chain of Custody for AI Evaluation Evidence

Every discipline that produces findings other people rely on has, eventually, been forced to solve the same problem. A forensic sample, a financial working paper, a clinical trial record, a calibration certificate — in each case the finding is worthless unless you can show where it came from, who handled it, and that it did not change quietly between the moment it was produced and the moment it was relied upon.

AI evaluation is arriving at this problem now, and mostly failing it. The dominant storage medium for AI assurance evidence is a spreadsheet on a shared drive, edited by an unknown number of people, superseded by a version emailed to a director, summarised in a slide that no longer matches either.

The question that exposes this is not technical. It is: two years from now, when someone asks how you concluded this system was safe, what exactly will you hand them?

What custody actually means

Chain of custody is not a product. It is a discipline, and it is defined by four properties of a record.

Provenance. Where did this come from? Which system, which version, which sample, on what date, drawn how. An evaluation score with no link to the artefact it scored is an orphan.

Attribution. Who did this? Not a team, not an account — a person, and, crucially, evidence that the person was qualified to do it. An assessment is only as good as the assessor's standing to make it, and the assessor's standing must be part of the record, not part of the folklore.

Sequence. What happened, in what order? A score changed after review is a different fact from a score entered once. The order in which things occurred is frequently the entire point of the enquiry.

Integrity. Can you show that the record has not been quietly altered since? This is the property everybody assumes they have and nobody has tested. A spreadsheet has none of it.

Every solid arrow is a handoff, and a handoff is where custody is lost. The dotted lines are the discipline that prevents it: each consequential action is written to an append-only trail as it happens, not reconstructed afterwards.

The four ways custody breaks

In practice, the break is almost never dramatic. Nobody forges a result. Custody is lost through ordinary, well-intentioned operational behaviour.

The silent edit. A score is corrected. The correction is right. But the record now shows only the corrected value, so a reader cannot tell whether the score was 2 all along or was a 4 until someone senior expressed a view. Both histories produce an identical spreadsheet, and only one of them is an evaluation.

The orphaned artefact. A report says the system scored well. Six months later nobody can identify which model version, which prompt, or which retrieval snapshot was assessed, because the configuration lived in an environment that has since been redeployed. The finding survives; the thing it was a finding about does not.

The uncredentialed assessor. The record says the review was performed. It does not say by whom, or on what basis they were competent to perform it. When the finding is challenged, there is no way to establish that the person who made the judgement was qualified to make it — and an assessment whose author cannot be qualified is an opinion.

The summarisation cascade. The scores become a report; the report becomes a slide; the slide becomes a sentence in a board pack; the sentence becomes a claim in a regulatory filing. At each hop, nuance and uncertainty are shed, because nuance does not survive compression. By the end, "76% pass rate on a sample of 400, with three open high-severity findings" has become "independently validated." Nobody lied. The chain simply had no mechanism for carrying the caveats forward.

That last failure is the most common and the most dangerous, and the only structural defence is that each layer of summary must point back at the layer beneath it — so that the reader of the sentence can, if they choose, reach the number, and the reader of the number can reach the score, and the reader of the score can see who gave it and what they were looking at.

Amend, never overwrite

The single most important operational rule in evidence handling is that records are appended, not edited.

A score that turns out to be wrong is not corrected in place. A new score is recorded, with a reason, by an identified person, at a known time, superseding the old one — which remains visible. A report that needs revision becomes version 2, which supersedes version 1 rather than replacing it. If someone withdraws a finding, the withdrawal is itself a recorded event with an author and a rationale.

This feels bureaucratic until the first time it saves you. An organisation that can show a finding was raised, challenged, re-examined, and revised — with each step attributed and timestamped — is demonstrating a functioning quality process. An organisation whose record shows only the final, tidy answer cannot distinguish itself from one that edited the inconvenient parts out, and under adversarial scrutiny it will not be given the benefit of the doubt.

Append-only records also make tampering detectable, which is a weaker and much more achievable property than making it impossible. When each entry in an audit log is chained to the one before it — each record carrying a hash of its predecessor — an alteration to any historical entry breaks the chain from that point forward. You cannot stop a sufficiently privileged actor from changing a database row. You can ensure that when they do, the log no longer verifies, and that is exactly the property an auditor needs: not a guarantee that nothing was changed, but a reliable way to tell.

The questions the record must answer

An evaluation record is adequate if, without asking anyone who was involved, a stranger can answer:

• What exactly was evaluated? Version, configuration, and the date it was frozen.
• What sample, drawn how? Including the population it was drawn from and the period it covers.
• Against what standard? Which rubric version, which taxonomy configuration, which severity definitions, which thresholds.
• By whom, and on what basis were they qualified? Named Evaluators, with their credentials part of the record.
• What did they see? The exact material presented at the moment of scoring.
• Where did they disagree, and how was it resolved? Agreement statistics, the disagreements, and the adjudication — who decided, and why.
• What was found, and what happened to it? Findings with severity, root cause, remediation status, and, where a finding was closed, the evidence that closed it.
• What changed after publication? Versions, supersessions, and the reason for each.

This is the standard SingleAxis is built to. Assignments, scores, gold-task results, adjudications, and report versions are captured as they happen; platform actions are written to a tamper-evident, hash-chained audit log; and the deliverable is a versioned Evidence Report (PDF or CSV) that states what was assessed, against which rubric and taxonomy configuration, by which credentialed Evaluators, with which findings — rather than a number whose origin has to be reconstructed from a calendar invite.

None of that is exotic. It is the same discipline a laboratory or an audit firm has practised for a century, applied to a class of evidence that is currently being kept in spreadsheets.

What to do tomorrow

Take the most important AI assurance claim your organisation has made in the last year — the one on a slide, in a filing, or in a contract — and try to walk it backwards. Find the sentence, then the report, then the scores, then the person, then the artefact that was scored.

Note where the trail goes cold. That gap is not a documentation problem to be tidied up later. It is the precise point at which your evidence stops being evidence, and it is where the questions will land.