In Mata v. Avianca (2023), a US federal judge sanctioned lawyers who submitted a brief containing six fictitious case citations generated by ChatGPT — complete with fabricated quotes and judicial names. The incident became the archetype for AI hallucination liability in professional services and has been followed by similar reports in the UK, Canada, and Australia.
Hallucination is not edge case
Large language models optimize for plausible text, not verified truth. When agents draft legal memos, medical summaries, financial analyses, or security assessments without grounding, they invent citations, statistics, and policy references that read authoritatively — and fail under scrutiny.
Global professional consequences
- US — courts sanction attorneys; state bars issue AI practice guidance
- UK — Solicitors Regulation Authority warns on unverified AI research
- Healthcare — reported misdiagnosis support from hallucinated literature
- Finance — analysts cited for AI-generated market data that did not exist
Enterprise controls
RAG with approved sources only — agents retrieve from verified corpora, not open generation for facts. Human-in-the-loop — consequential outputs require professional sign-off. Compliance tests — citation validation suites before deploy. Disclosure — transparency when AI assisted a customer-facing or regulatory document.
Governance intake question
Does this agent produce externally binding or professionally certified output? If yes, risk tier increases; Regal AI assigns stricter controls. Hallucination is a model behavior — liability is an enterprise choice to deploy without governance.