The AI Compliance Audit Checklist

An auditor is coming to review your AI systems. What will they ask? What will they expect to see?

Having worked with compliance teams across industries, we've compiled what auditors actually look for. Use this checklist before they arrive—not after.

The Auditor's Mindset

Auditors aren't trying to trick you. They're trying to answer three questions:

1. Do you know what your AI systems are doing?
2. Can you prove it?
3. Do you have controls in place?

If you can demonstrate all three, audits become straightforward. If you can't, they become painful.

Pre-Audit Preparation

Documentation Checklist

• AI System Inventory

  • Complete list of all AI systems in production
  • Classification by risk level (EU AI Act categories)
  • Business owner for each system
  • Technical owner for each system
  • Deployment date and version history

• System Documentation

  • Technical architecture diagrams
  • Data flow documentation
  • Training data sources and provenance
  • Model selection rationale
  • Known limitations and failure modes

• Governance Documentation

  • AI ethics policy
  • Acceptable use policy
  • Change management procedures
  • Incident response procedures
  • Human oversight protocols

Evidence Checklist

• Audit Logs

  • Complete action logs for audit period
  • Tamper-evident or immutable storage
  • Retention meets regulatory requirements
  • Logs include actor, action, timestamp, context

• Decision Records

  • High-risk decisions documented
  • Reasoning captured where applicable
  • Human review records for escalations
  • Override history with justifications

• Performance Records

  • Accuracy metrics over time
  • Error rates and types
  • Model drift detection results
  • Bias testing results

During the Audit

Common Questions and How to Answer Them

Q: "Can you show me every decision this AI made for Customer X?"

You need: Searchable audit logs filtered by customer ID, returning all actions with full context.

{
  "query": {
    "customer_id": "customer-x",
    "date_range": "2024-01-01 to 2024-12-31"
  },
  "results": [
    {
      "timestamp": "2024-03-15T14:23:00Z",
      "action": "credit_decision",
      "decision": "approved",
      "reasoning": "Score 720, income verified, DTI 28%",
      "human_review": "none_required"
    }
  ]
}

Q: "Why did the system make this specific decision?"

You need: Context capture at decision time, including inputs, reasoning, and confidence.

Q: "Who has access to modify this system?"

You need: Access control lists, role definitions, and change logs.

Q: "How do you know the system isn't biased?"

You need: Bias testing methodology, test results, remediation actions taken.

Q: "What happens when the system fails?"

You need: Incident logs, escalation records, fallback procedure documentation.

The Five Evidence Types

Auditors look for five types of evidence:

1. Design Evidence

Proves you thought about compliance before deployment.

• Risk assessments conducted before launch
• Compliance requirements incorporated into design
• Testing plans that include compliance scenarios
• Sign-offs from legal, compliance, and business

2. Operating Evidence

Proves the system operates as designed.

• Audit logs showing normal operation
• Performance metrics within expected ranges
• Alert and response records
• Scheduled review documentation

3. Testing Evidence

Proves you verify compliance regularly.

• Bias testing results (quarterly minimum)
• Accuracy benchmarking results
• Edge case testing results
• Penetration testing results (where applicable)

4. Incident Evidence

Proves you handle problems appropriately.

• Incident tickets and resolution records
• Root cause analysis documentation
• Remediation action tracking
• Post-incident review notes

5. Oversight Evidence

Proves humans remain in control.

• Human review sampling records
• Override decision logs
• Escalation records
• Regular management review meeting notes

The Audit Trail Structure

A compliant audit trail captures:

Minimum Fields Per Action

Extended Context for High-Risk Actions

For actions that auditors will scrutinize, capture:

{
  "extended_context": {
    "inputs": {
      "credit_score": 720,
      "income": 85000,
      "debt_to_income": 0.28
    },
    "model_version": "credit-model-v3.2.1",
    "confidence": 0.94,
    "alternative_decisions": [
      {"decision": "manual_review", "score": 0.04},
      {"decision": "declined", "score": 0.02}
    ],
    "reasoning": "Score exceeds threshold (680), DTI below limit (0.43)",
    "policy_version": "credit-policy-2025-01"
  }
}

This level of detail answers the "why" question that auditors always ask.

Industry-Specific Requirements

Financial Services (SOC 2, SOX)

• Access controls documented and enforced
• Change management procedures followed
• Data encryption at rest and in transit
• Segregation of duties demonstrated
• Regular access reviews conducted

Healthcare (HIPAA)

• PHI access logging complete
• Minimum necessary principle applied
• Business associate agreements in place
• Breach notification procedures documented
• Regular risk assessments conducted

EU AI Act (High-Risk Systems)

• Risk management system documented
• Training data governance demonstrated
• Technical documentation complete
• Automatic logging implemented
• Human oversight mechanisms in place

Red Flags That Trigger Deeper Review

Avoid these audit triggers:

• Incomplete logs - Gaps in the audit trail suggest data integrity issues
• Unexplained decisions - "The model said so" is not acceptable
• No human oversight records - Suggests lack of control
• Missing documentation - Suggests immature governance
• Inconsistent processes - Suggests lack of standardization

Post-Audit: Common Findings

Typical audit findings and how to prevent them:

The Empress Compliance Toolkit

Empress provides audit-ready infrastructure:

• Complete audit logs in xAPI format
• One-click compliance reports for common frameworks
• Tamper-evident storage for regulatory requirements
• Search and export for auditor requests
• Pre-built compliance dashboards for ongoing monitoring

When auditors arrive, you export a report. That's it.

Audit Readiness Checklist

Run this monthly:

• All AI systems inventoried and classified
• Audit logs complete for the period
• No gaps in logging coverage
• High-risk decisions have extended context
• Human oversight records current
• Documentation matches reality
• Access controls reviewed
• Bias testing current
• Incident records complete

If you can check all boxes, you're audit-ready. If not, you know where to focus.

Compliance isn't about the audit. It's about operating responsibly every day. The audit just verifies what you're already doing.