AI Compliance Liability: The Hidden Enforcement Risk
AI Compliance Systems and Legal Liability
Why Black-Box AI Creates Legal Risk
An FCA team investigates a non-compliant financial promotion. They contact the firm: “What analysis did you perform before approval?”
The firm’s answer: “Our AI system assessed it and approved it.”
The FCA’s follow-up: “Can you show us the analysis? What regulatory requirements did you review?”
The firm’s response: “The system produces only approval or rejection. There’s no reasoning recorded.”
Here’s the thing. That answer creates a much larger problem than whether the promotion itself was compliant.
Courts on accountability and explainability
Courts have spent 160 years making one point consistently: deploying a system doesn’t remove your responsibility for what it does.
The principle shows up everywhere. In Braganza v BP Shipping, the Supreme Court held that when you make discretionary decisions, they have to be rational, evidence-based, and explicable. You can’t hide behind systems. You can’t say “the algorithm decided.” A court looking at your decision-making will ask: Did you understand what you were deciding? Was your process based on evidence? Can you explain why you reached that conclusion?
In Bridges v South Wales Police, the Court of Appeal applied this directly to algorithmic systems. The court didn’t say facial recognition technology is inherently bad. It said the governance framework was inadequate. When you deploy algorithmic decision-making, you need documented rules governing how the system is designed, deployed, and overseen. Vague policies and hidden systems aren’t good enough.
The pattern is consistent across cases: deploying technology doesn’t displace your responsibility. It sharpens the need for governance around the system.
That principle matters directly to what’s happening with AI compliance approvals right now.
FCA Rules on Automated Compliance Tools
The FCA requires firms to “take reasonable care to organise and control its affairs responsibly and effectively, with adequate risk management systems.” This isn’t abstract. It applies directly to automated compliance tools.
Under SYSC 3, you have to establish and maintain:
governance and accountability for how the tool is used
risk assessment, monitoring, and challenge of the system’s performance
records sufficient to evidence that compliance decisions were actually reviewed and controlled
Here’s the enforcement reality: if an AI approval system can’t produce these materials, the firm can’t demonstrate adequate systems and controls. That’s not just a technical problem. That’s a regulatory exposure.
The defect triggers the investigation. Governance failure drives the enforcement.
This is the insight most institutions miss. Regulators don’t focus on outcomes. They focus on systems. They don’t require perfect decisions. They require demonstrable governance around the systems that produce decisions.
When a compliance decision looks wrong, an FCA investigation starts. But when a firm can’t show how it reviewed requirements, applied logic, or exercised human judgment—that’s when enforcement happens.
Case studies: Starling Bank and Nationwide fines
The pattern is clear in recent FCA cases.
Starling Bank received a £29 million fine for financial sanctions screening controls that were, as the FCA put it, “shockingly lax.” The regulator criticised inadequate monitoring of automated decisions, staff unable to override automated rejections, and insufficient record-keeping to show how decisions were made. The enforcement focus was on the adequacy of the firm’s systems and controls—whether known weaknesses had been identified and remediated.
Nationwide Building Society received a £44 million fine for inadequate AML systems. The FCA’s statement was specific: the firm “was aware of weaknesses in its systems but failed to remediate them.” The regulator wasn’t penalising specific AML violations. It was penalising the firm for deploying automated systems without adequate oversight, monitoring, or challenge mechanisms.
Both cases teach the same lesson: inadequate governance of automated systems is itself a compliance concern, independent of whether specific decisions were defective.
When an FCA team investigates your AI approval system, they ask for:
compliance policies and procedures
system documentation and design specifications
validation and testing records
decision logs and audit trails
evidence of human oversight and review
monitoring and performance data
If you can’t produce these materials, the FCA concludes you operated inadequate systems and controls. The absence of documentation becomes the evidence.
Governance failures that trigger investigations
Under UK GDPR Article 22, individuals have the right not to be subject to decisions based solely on automated processing that have legal or similarly significant effects. Where automated decisions occur, you need to provide transparency about how decisions are made.
Article 22 matters most where an AI-driven process directly determines a decision affecting an individual automated rejection of an application, for example, or an automated credit determination without meaningful human review. In those cases, if you can’t explain the basis of the automated output, you’re facing serious compliance difficulty.
For financial promotions specifically, the requirement isn’t Article 22. It’s the basic governance standards under SYSC and COBS. A firm approving promotions through a black-box system can’t demonstrate it reviewed the relevant regulatory requirements or exercised governance over the approval process. That’s a breach of approval responsibility under FSMA 2000.
What enforcement investigators actually look for
In practice, investigators don’t begin with abstract questions about AI ethics. They begin with files, controls, logs, validation, and accountability.
When regulators investigate AI compliance systems, they examine:
System design and validation
How was the system developed and tested before deployment?
What data was used for training?
What known limitations or performance risks exist?
Governance and oversight
Who is accountable for the system?
What monitoring, challenge, and escalation mechanisms exist?
What board-level oversight occurs?
Decision documentation
Can you produce decision logs showing what inputs, parameters, and versions were used?
Can you explain, for individual decisions, what the system assessed and why?
Is there evidence of human review or challenge?
Regulatory alignment
Does the firm understand which regulatory requirements apply to the system?
Can it demonstrate those requirements were incorporated into system design?
Does it maintain documentation showing compliance with those requirements?
If you can’t answer these questions, particularly the decision documentation and regulatory alignment questions, you’re facing significant enforcement exposure.
Why the legal frameworks point in the same direction
This is where it gets interesting. Multiple legal regimes converge on exactly the same governance expectation.
Common law requires rational, explicable decision-making. If you deploy a system, you have to be able to explain how it reached conclusions.
FCA regulation requires adequate systems and controls. An unexplainable system can’t demonstrate adequacy.
GDPR and data protection require explainability for automated decisions affecting individuals. If your system can’t explain its logic, you may violate Article 22.
Emerging AI governance frameworks expect documented risk management, technical specifications, and human oversight for high-risk systems.
These aren’t contradictory requirements stated in different languages. They’re the same requirement converging from multiple directions. A black-box approval tool creates the same basic problem across all four frameworks: the firm can’t show how the decision was governed, assessed, or explained.
That convergence is significant. It means you’re not managing a single regulatory risk. You’re managing a structural governance problem that every framework recognises.
The governance answer
The answer isn’t to abandon AI. It’s to govern it properly.
That means structured analysis rather than binary outputs. Show requirements assessed, reasoning applied, confidence levels assigned. That means meaningful human challenge rather than passive sign-off. That means records showing what was assessed, by whom, and on what basis.
The firms most exposed aren’t the ones using AI. They’re the ones using AI without building an evidential record around it.
🗂 Case closed: Firms remain responsible for the systems they deploy. Accountability doesn’t disappear inside a black box.