What is an AI audit evidence pack?
An AI audit evidence pack is the set of records an organization produces to answer somebody else's questions about how it uses AI: what is in use, what rules govern it, who is accountable, what a given system did, and what changed. It is normally assembled in response to a request rather than maintained as a standing artifact, and that is the single fact that determines how long it takes to produce and how well it holds up.
What a pack usually contains
The list of AI systems in use, and what each is used for. Whatever written rule governs that use. A named person accountable for each system. Records of individual outputs or decisions, with enough attached to say which tool and which version produced them. A record of changes: version moves, scope changes, vendor features that appeared. And evidence that whatever review cycle you describe actually ran.
The exact contents vary with who is asking. A customer's security review wants current state and a short answer. An auditor wants a period that has already ended and will read the artifacts rather than your summary. An insurer usually wants the narrowest slice of all, and wants it in their own form.
What does not vary is the underlying question, which is whether each item can be traced back to something other than a person's account of it.
Why the pack is usually assembled, and what that costs
Most organizations have the raw material and have never gathered it. The pack is built the first time somebody asks, by one person, under a deadline, from systems that were not designed to be read this way. That first assembly is where the surprises live: the quarter that aged out of a log, the approver who left, the tool nobody can export from, the screenshot whose source nobody can now identify.
The cost is paid again on the next request, because the assembly produced a document rather than a capability. Two years of questionnaires can pass without the second one being any cheaper than the first.
The alternative is not a bigger document. It is a smaller one that is a view over records the work already produces, so that assembling it collects rather than creates.
Links, and the difference a shared identifier makes
A pack is not a folder of items. It is a set of links: this output came from that input, under that policy version, approved by that person. The items on their own answer almost nothing.
Each link is carried by one of three things. A shared identifier present in both records, which anybody can follow. A person who remembers, which is an answer for exactly as long as that person is reachable. Or nothing, in which case the link does not exist and the two records merely coexist.
This is the distinction the shipped Audit Evidence Pack Assay is built on, and it reports a link resting on a person's assertion as needing a witness, because that is what it is. The reason it matters for assurance rather than only for audits is that witnesses become unavailable on a schedule nobody controls.
What a pack is not
It is not proof of compliance with anything. It is the material somebody else uses to reach their own conclusion, and a complete pack can support an unfavourable one.
It is not a report. Summarising your own evidence is useful and is a separate artifact; a reviewer who is given only the summary will ask for what it was built from.
It is not something a template can supply. A template can tell you what is usually asked for. What is in your systems, and whether it can be linked, is a fact about your systems.
Common questions
- How long should an evidence pack take to produce?
- The number is less important than whether you know it. An organization that has timed a real production knows what a request costs it and can commit to a deadline; one that has only estimated finds out during the request. The gap between the two is usually large, because the estimate is made by the person who knows where everything is.
- Should the pack be kept ready, or built on request?
- Neither, exactly. Keeping a pre-built pack ready means maintaining a second copy of everything, which decays as fast as anything else. The durable arrangement is that the underlying records are kept in a form the pack can be generated from, so that producing it is a collection task rather than an authoring one.
- Does a screenshot count as evidence?
- It counts, and it is weak, because it carries no provenance: six months later nothing distinguishes a screenshot taken from a live system from one taken from a test account or edited afterwards. An export from the system that produced the record answers the same question once instead of raising a second one.
Find out where you stand
The assessment on this site asks up to 24 questions about what makes your evidence stop being reproducible, and returns a score, a verdict and, where a control is short, the one thing to fix first. It names no framework, it takes about ten minutes, and where nobody outside asks, the two questions about somebody else's deadline and somebody else's reader are dropped rather than scored against you.
It is a self-assessment of whether what you can show today would still be showable later, not an audit, not a certification, not a conformity assessment and not legal advice. It says nothing about whether the practice it describes is adequate.
Run the assessment