Building an AI Assurance Function: Roles, Skills, and Org Design
Building an AI Assurance Function: Roles, Skills, and Org Design
Most organisations deploying AI at scale have, by now, appointed someone to be responsible for AI risk. Fewer have thought carefully about where that person sits, what they are measured on, and what happens when they say no to a launch that the business has already announced.
Those three questions determine whether you have an assurance function or a compliance decoration. Nothing about the person's competence, the framework they adopt, or the quality of their policy documents can compensate for getting them wrong.
The independence problem, stated plainly
An assurance function exists to tell the organisation things it does not want to hear, at moments when it is expensive to hear them. That is the entire job. Every other activity — writing policy, running training, maintaining a register — is preparation for the one moment when the function has to stand in front of a launch and say the evidence is not there.
Which means the design question is not "who is good at AI risk?" It is: when the function's finding is inconvenient, what mechanically happens?
If the head of assurance reports to the executive whose bonus depends on the launch, the answer is known. Not because anyone is corrupt, but because that is what reporting lines do. The finding gets softened in the retelling, the severity is negotiated downward, the remediation becomes a fast-follow, and the fast-follow becomes a backlog item. Nobody experiences themselves as having done anything wrong.
This is the oldest lesson in the assurance professions and it is being relearned, expensively, in AI. Financial audit is independent of the finance team. Clinical trial monitoring is independent of the investigator. Safety engineering is independent of the programme it certifies — and where it has not been, the accident reports say so.
Three lines, applied to AI
The three-lines model is unfashionable and correct. It survives because it encodes one idea: the people who do the thing, the people who set the rules for the thing, and the people who check the thing must not be the same people.
Two features of this diagram do the real work.
The first line owns the risk. Assurance does not own it, and this is constantly misunderstood. A governance function that accepts ownership of AI risk has quietly relieved the product team of it, and the product team will behave accordingly — treating risk as somebody else's gate to pass rather than a property of the thing they are building. The correct posture is that the team that ships the system owns its consequences, and assurance's job is to make the state of that risk visible and undeniable.
The third line reports past the executive whose work it is checking. If that dotted line to the board does not exist, the whole structure collapses into a first line with extra paperwork.
Who you actually need
Assurance is not one skill. The most common staffing failure is hiring three people with the same background — usually risk-and-policy — and discovering that nobody can read a model card, let alone design a test.
The evaluation lead. Owns the methodology. Understands sampling, inter-rater agreement, rubric design, and the difference between an error rate and a finding density. This is a measurement discipline, and it is the role most often filled by someone who has never designed a study.
Domain assessors. People qualified in the field the AI operates in — clinicians, underwriters, lawyers, engineers. They are the only ones who can tell a subtly wrong answer from a right one, and no amount of AI expertise substitutes. This is the scarcest resource in the entire function and the one that gets cut first.
The adversarial capability. Someone whose job is to make the system fail: prompt injection, memory contamination, scope escape, jailbreaks, data exfiltration. Temperamentally distinct from the assessors, and rarely the same person.
The technical reader. Can read the system, not just the docs. Can determine what the agent's tools actually permit, whether the guardrail is enforced in code or requested in a prompt, and whether the logging captures what the policy claims it captures.
The regulatory translator. Maps obligations to controls and controls to evidence, in both directions. Their most valuable output is telling the business which of its evidence gaps are real exposures and which are anxiety.
The programme owner. Runs the register, the cadence, the triggers, and the reporting line to the board. Unglamorous, and the function does not function without it.
Most organisations need fractions of these, not full-time headcount for each. What they cannot do is collapse them into one person, because the aggregate is not a job description — it is a portfolio of genuinely different skills, and hiring for it produces either a policy writer who cannot test or an engineer who cannot report.
Where it sits, and what it is measured on
Two structural decisions follow.
Reporting line. Assurance must have a path to the board or risk committee that does not pass through the executive accountable for delivering the AI. Any other arrangement means the function's escalation route runs through the person it may need to escalate about.
Metrics. What you measure the function on determines what it becomes. Measure it on throughput — evaluations completed, reviews closed — and it will become a rubber stamp with a good SLA. The metrics that keep it honest are uncomfortable ones: findings raised and their severity distribution, remediation closure time, the rate at which deployed systems produce incidents the evaluation did not predict, and the proportion of production AI covered by current evidence. That last one is the number a board should ask for and almost never does.
The build/buy line
Some of this work must be internal. Some of it is structurally better done outside.
Internal ownership is right for the risk register, the policy and taxonomy, the deployment gates, the monitoring, and the incident response — anything that requires standing context and constant presence.
External evaluation is structurally better wherever the value of the finding depends on the assessor having no stake in the answer, wherever the domain expertise is expensive to hold permanently, and wherever an outside party will be reading the result. A team cannot audit itself, and an internal team that reports to the same executive as the builder is closer to self-audit than anyone in the meeting will admit.
That is the case for independent, human-led evaluation under a framework: credentialed Evaluators, a rubric that encodes your policy, agreement measured between assessors rather than a single opinion, disagreements adjudicated rather than averaged, and a versioned Evidence Report with findings, severities, and root causes that goes to your third line and your board. It is not a replacement for an internal function. It is the thing your internal function commissions when the answer needs to be believable to someone who does not work for you.
What to do tomorrow
Draw your actual reporting lines — not the ones in the policy document. Find the person who would have to say no to your most commercially important AI launch, and trace their line upward. If it terminates in the executive who owns that launch, you have found the flaw, and it is not fixable with a better framework.
Then ask one question at your next risk committee: what proportion of the AI systems we have in production have current, independent evidence that they work? The silence that follows is the size of the function you need to build.