Why Questionnaires Fail for AI Compliance

The default approach to AI compliance is the questionnaire: consultancies send spreadsheets, GRC platforms present forms, internal teams build checklists. For traditional software, this works well enough. For agentic AI systems, it fails in predictable ways.

What Respondents Don't Know

A questionnaire relies on the respondent knowing the answer. For agentic systems, many compliance-relevant facts are invisible to the person filling in the form:

• Stop mechanisms: the compliance officer doesn't know whether the orchestration layer supports mid-chain interruption, at what granularity, or what happens to in-flight actions.
• Data flows: in an agentic system, data flows are dynamic. Agent A passes context to Agent B, which calls a tool that queries a database. The form respondent sees an architecture diagram, not the actual runtime paths.
• Logging granularity: "Yes, we have logging" doesn't satisfy Article 12. Whether the logs can reconstruct the full decision chain is a technical question answerable by inspecting code.
• Tool access: dynamic tool discovery means the answer changes at runtime. A questionnaire captures a snapshot that may be wrong by tomorrow.

False Compliance

The greater risk is confident wrong answers. Common false positives:

• "We have logging" (it logs HTTP requests, not agent decision reasoning)
• "A human reviews outputs" (the agent acts before review happens)
• "We have documentation" (it describes intended behaviour, not actual behaviour)
• "We don't process personal data" (the agent's tool calls hit a CRM containing personal data)

Each creates a finding that appears compliant on paper and fails under regulatory scrutiny.

What Technical Scanning Catches

Code inspection produces evidence-based findings questionnaires cannot: missing stop mechanisms (from orchestration code), dynamic agent spawning (factory patterns, unbounded recursion), autonomous decision paths without human checkpoints (input-to-action tracing), prohibited instruction patterns (system prompt scanning against Article 5), logging coverage gaps (configuration inspection against Article 12), GPAI model identification (triggering Articles 51-55), and full tool inventories extracted from code definitions.

Scan First, Ask Second

1. Technical scan: extract what can be determined from code (agents, models, tools, logging, orchestration, data flows, stop mechanisms).
2. Auto-evaluate: apply compliance rules to the evidence collected.
3. Targeted questions: generate questions only for what the scan couldn't determine (decision domains, intended purpose, organisational context).
4. Human confirmation: every finding requires explicit sign-off before entering the report.

Questionnaires remain appropriate for organisational controls (governance, policies, training, DPO appointments), intended purpose, and risk context. The problem is using them for the entire assessment, particularly the technical portions where penalty exposure is highest.

Part of our guide: EU AI Act compliance for UK organisations.