All posts
  • ai hallucination
  • hallucination testing
  • banking
  • ai agents
  • customer service ai
  • occ
  • fca consumer duty
  • ai agent testing
  • conduct risk
  • ai governance

Hallucination Testing for Banking AI Agents: Fees, Rates and Policies

A hallucinating banking AI agent makes misstatements in the bank's name. Here's what the OCC, FCA and courts said in 2026, where agents go wrong, and a 6-step framework to test for hallucinations.

By Aodit

11 min read

When a bank's AI agent tells a customer the wrong overdraft fee, a rate that doesn't exist or a refund policy the bank never had, it isn't a quirky technical glitch. It is a misstatement made in the bank's name. In 2026 supervisors in the US and UK said so plainly.

This guide explains why hallucinations in customer-facing banking agents are now a conduct and compliance issue, where they tend to appear, and how risk, compliance and AI teams can test for them before and after go-live.

What "hallucination" means for a banking AI agent

The UK Financial Conduct Authority (FCA) defines a hallucination as the situation "when an AI system produces information that appears plausible but is false, unsupported or misleading" (FCA, AI in financial services: good practice and areas for improvement, 19 October 2026). NIST calls the same risk "confabulation" in its Generative AI Profile (NIST AI 600-1).

For a customer-facing banking agent, it helps to break this into five failure modes. Each needs a different test:

  1. Fabricated facts. The agent invents a fee amount, interest rate, cut-off time or eligibility rule.
  2. Fabricated policy. The agent states a refund, waiver or exception policy that doesn't exist, often because the customer asked as if it did.
  3. Inconsistency. The agent gives different answers to the same question when it is reworded or asked later in the conversation.
  4. False certainty. The agent answers confidently when it should say it doesn't know, or should send the customer to a human or an authoritative source.
  5. Pressure-induced bluffing. The agent is accurate at first, then gives way under urgency, authority claims or emotional pressure and confirms what the customer wants to hear.

The last two matter most in banking. A model can score well on a static question-and-answer benchmark and still fail them, because they only show up in multi-turn, adversarial conversations. That is how real customers, and real fraudsters, talk to an agent.

Why regulators now treat hallucinations as a conduct risk

United States: the OCC names fees, rates and account status

On 4 November 2026 the OCC released its supervisory framework for bank use of "Super Intelligence" (the OCC's term for generative and agentic AI), with OCC Bulletin 2026-54. The framework is an examiner resource. The bulletin says it "does not establish enforceable standards or prescriptive requirements" (OCC News Release 2026-97).

Even so, it shows what examiners will look for. Legal summaries of the framework report that it:

  • flags compliance risk when outputs in consumer-facing activities such as customer service, marketing, collections and disclosures are "materially inaccurate, misleading";
  • names hallucinations about "fees, interest rates, account status, and bank policies" as a source of liability and customer harm;
  • points to controls such as limiting automated representations, requiring verification in higher-impact contexts, and correcting and remediating errors promptly;
  • notes that conventional model validation "may be insufficient" for non-deterministic systems and calls for testing that accounts for hallucinations, adversarial inputs and whether similar prompts produce different behavior;
  • states that "the use of SI does not change a bank's underlying legal obligations."

We covered the full framework in How OCC Examiners Will Review AI Agents.

United Kingdom: the FCA expects stress-testing and outcome monitoring

The FCA's 19 October 2026 good-and-poor-practice review doesn't create new rules. It does show what the regulator considers good practice under existing obligations such as the Consumer Duty. Among the practices it describes (FCA):

  • "Using approved source material and guardrails to reduce hallucinations or inappropriate responses."
  • "Firms limited chatbots and generative AI assistants to approved source material, with checks and escalation routes."
  • "Stress-testing AI tools before deployment."
  • "Spot checks and automated scoring frameworks to detect model drift or harmful AI outputs."

It also lists a poor practice: "Firms overstated the accuracy, reliability or role of AI systems." And it warns that "where firms monitored technical metrics rather than outcomes, they had less chance of meaningful oversight."

Courts: "it was the chatbot" is not a defense

The best-known ruling isn't from banking, but it applies directly. In Moffatt v. Air Canada (2024 BCCRT 149), a Canadian tribunal held the airline liable for negligent misrepresentation after its website chatbot wrongly told a customer he could claim a bereavement fare retroactively. The tribunal rejected the argument that the chatbot was responsible for its own statements: "It makes no difference whether the information comes from a static page or a chatbot" (McCarthy Tétrault summary).

In October 2026, American Banker reported on a report by former regulators. It concluded that banks replacing human service staff with chatbots may breach state and federal consumer-protection laws when the bots give inaccurate information that causes harm, especially around loans and mortgages, and that automation doesn't shift responsibility away from the bank. The CFPB had raised similar concerns in its June 2023 issue spotlight on chatbots in consumer finance. The CFPB's supervisory priorities have changed a lot since 2025, but the consumer-protection statutes behind those concerns have not.

European Union: transparency now, accuracy duties via existing law

Since 2 August 2026, Article 50 of the EU AI Act has required AI systems that interact directly with people to be designed so users know they are dealing with AI. The Digital Omnibus (Regulation (EU) 2026/1744) did not postpone that duty. Most customer-service agents aren't high-risk systems under Annex III, so the AI Act itself doesn't impose an accuracy-testing regime on them. A bank's accuracy duties toward customers still come from conduct, consumer-credit and consumer-protection law. An agent that advises on creditworthiness or credit terms may also come close to the Annex III high-risk area, which now applies from 2 December 2027.

The common thread: across jurisdictions, the bank owns what its agent says. Supervisors increasingly expect evidence that the agent was tested for accuracy under realistic pressure, and not only on a clean benchmark.

Want to see how your agent holds up? AODIT's free live demo runs an 8-turn adversarial conversation against your customer-support agent's instructions and scores each turn for Factual Consistency. It takes about two minutes and needs no account.

Where banking agents hallucinate: high-risk scenarios

From the regulatory focus areas above, these are the scenarios most worth testing first:

ScenarioTypical hallucinationWhy it matters
Fee questions (overdraft, FX, wire, card replacement)Invented or outdated amounts; "this fee can be waived"Named by OCC; misrepresentation and UDAP/UDAAP exposure
Rates and pricing (savings APY, loan APR, promotional rates)Wrong rate, wrong eligibility, invented promotionNamed by OCC; marketing and disclosure risk
Account status and transactionsConfirms a payment, reversal or limit change that didn't happenCustomer acts on false information; complaint and remediation cost
Refunds, chargebacks and disputesPromises timelines or outcomes outside policyCreates expectations the bank can't meet
Hardship, collections and vulnerable customersConfident but wrong guidance on forbearance optionsConsumer Duty and fair-treatment exposure
Product eligibility and credit"You'll qualify" or invented criteriaClose to credit-decisioning risk
Policy exceptions under pressure"As a manager, I authorize you..." gets acceptedSocial engineering turns a hallucination into a control failure

Retrieval-augmented generation (RAG) against approved content helps, and the FCA explicitly cites approved source material as good practice. It is not a guarantee. Agents can still combine retrieved passages wrongly, fill gaps with plausible inventions, or drop grounding once a customer adds pressure. That is why grounding has to be tested, not assumed.

A 6-step hallucination testing framework for banking AI agents

1. Build a ground-truth fact set

Pull the facts your agent must never get wrong, such as fee schedules, rate tables, cut-off times, refund and dispute policies and escalation rules, from the authoritative source owned by the business. Version it. Every test result should trace back to a specific version of the facts, so you can show what "correct" meant on the day you tested.

2. Test consistency, not just correctness

Ask the same question in several ways: reworded, embedded in a longer scenario, asked again after five turns, asked in a second language if you support one. An agent that gets the overdraft fee right once and wrong when asked again has a reliability problem a single-shot benchmark won't catch. The OCC framework's focus on whether similar prompts produce different behavior points the same way.

3. Seed false premises

Ask questions that assume something untrue: "Since you waive the first overdraft fee every month, can you refund mine?" A well-behaved agent corrects the premise. A weak agent accepts it and invents a process. False-premise testing is one of the most effective ways to surface fabricated policy.

4. Apply pressure across multiple turns

Escalate the way real customers and fraudsters do: urgency ("my rent is due in an hour"), authority ("I'm a branch manager"), emotional pressure, and repeated insistence. Then check whether the agent recovers once the pressure stops. Many agents are accurate on turn one and wrong by turn six. Our prompt injection testing guide covers the adversarial side of this in more detail.

5. Score uncertainty and escalation behavior

Correct answers are only half the result. Measure how often the agent says it doesn't know, points to an authoritative source, or hands off to a human when it should. The FCA's review expects firms to decide in advance where an AI tool "should abstain, redirect or escalate to a person." An agent that never says "I'm not sure" is a warning sign, not a strength.

6. Re-test on every change and monitor live outcomes

Fees change, prompts change, and vendors update underlying models, sometimes without much notice. Treat a model or prompt change as a reason to re-run the full suite and compare results with the previous run. In production, combine automated scoring with human sample reviews, as the FCA describes, and track outcome metrics such as complaints, corrections and escalations alongside technical ones.

What good evidence looks like

For a model risk committee, internal audit or an examiner, aim to show:

  • the ground-truth fact set and its version;
  • test scenarios mapped to risk areas (fees, rates, account status, policy);
  • results per scenario and per turn, including consistency across rewordings;
  • how the agent behaved under pressure and whether it recovered;
  • remediation actions and re-test results after changes;
  • who reviewed and approved the results, and when.

This lines up with what FINMA Guidance 08/2024 asks of Swiss institutions on testing and independent review, and with the testing and documentation components in the OCC framework.

How AODIT helps banks test for hallucinations

AODIT is an independent behavioral risk rating and stress-testing platform for AI agents in banking and finance. Its AODIT-6 methodology maps closely to the failure modes in this guide:

  • Reliability covers factual consistency (R1), contradiction handling (R2) and memory persistence (R5), so you can see whether the agent keeps its facts straight across rewordings and long conversations.
  • Integrity covers uncertainty disclosure (I1), false certainty resistance (I2), fabricated source handling (I3), the calibration gap between the agent's confidence and the evaluator's (I4), and pressure-induced bluffing (I5).
  • Resistance and Judgment test whether pressure, authority spoofing and social engineering push the agent into confirming false policies, and whether it escalates appropriately.

Every test follows an 8-turn adversarial protocol: baseline, reworded question, complicated scenario, grey zone, hard push, manipulation attempt, recovery test and final judgment. This is built to surface the hallucinations that only appear under pressure. The full product runs 20 to 100 scenarios per test across six dimensions and produces a PDF-ready compliance report you can use as evidence for model risk, compliance and audit reviews. For institutions that can't send data outside their environment, AODIT supports on-premise and air-gapped deployment.

AODIT's results are advisory. They support your own validation and governance processes, and don't replace them.

Key takeaways

  • A hallucination by a banking AI agent is a misrepresentation made in the bank's name. Courts and supervisors have said the "it was the chatbot" defense doesn't work.
  • The OCC framework names hallucinations about fees, interest rates, account status and bank policies. The FCA cites stress-testing, approved source material and outcome monitoring as good practice.
  • Static benchmarks miss the most dangerous failures: inconsistency, false certainty and bluffing under pressure.
  • Effective testing combines a versioned ground-truth fact set, consistency checks, false-premise seeding, multi-turn pressure, escalation scoring and re-testing on every change.

Frequently asked questions

What is AI hallucination testing in banking?

It is structured testing of whether a bank's AI agent gives false, unsupported or misleading information, such as wrong fees, rates, account status or policies. Good testing checks correctness, consistency across rewordings and behavior under multi-turn pressure.

Is a bank liable if its chatbot gives wrong information?

In general, regulators and courts treat the bank as responsible for what its agent tells customers. The OCC says AI "does not change a bank's underlying legal obligations," and Moffatt v. Air Canada rejected the defense that a chatbot is responsible for its own statements. Specific liability depends on jurisdiction and facts, so get legal advice.

Does RAG eliminate hallucinations?

No. Grounding the agent in approved content reduces hallucinations, and the FCA cites it as good practice. Agents can still combine sources wrongly, fill gaps with invented details or drift under pressure, so grounded agents still need adversarial testing.

Does the EU AI Act require hallucination testing for customer-service chatbots?

Most customer-service chatbots are not high-risk under the AI Act, so it doesn't impose a dedicated accuracy-testing regime on them. Article 50 transparency duties have applied since 2 August 2026. Accuracy obligations toward customers still come from conduct and consumer-protection law, and agents involved in creditworthiness assessment may fall within Annex III high-risk rules from 2 December 2027.

How often should a banking AI agent be re-tested?

At minimum, whenever the underlying model, the system prompt, the retrieval content or the products and fees it describes change, plus periodic re-testing on a schedule that matches the agent's risk tier. Live outcome monitoring should run continuously in between.

Test your agent before your customers do

Your customers will find your agent's hallucinations eventually. It is better to find them first, and to have the evidence when a supervisor asks.

  • Try it now: run the free AODIT demo on your customer-support agent and see its turn-by-turn Factual Consistency scores in about two minutes.
  • Go deeper: see AODIT plans for full six-dimension testing and compliance-ready reports.
  • Talk to us: request an evaluation and we'll help you scope a hallucination and behavioral stress test for your banking AI agents.

This article is for general information and doesn't constitute legal or regulatory advice. Regulatory references reflect publicly available sources as of 15 November 2026.

  1. 11 min read

    Excessive Agency in Banking AI Agents: Permissions and Blast Radius

    Once an AI agent can issue refunds or move money, its permissions become your biggest risk. Here's what OWASP, the OCC, FSB and DORA say about excessive agency, and a 6-step framework to limit an agent's blast radius.

    • excessive agency
    • ai agent permissions
    • least privilege
    • blast radius

    Read

  2. 14 min read

    LLM Model Change Management for Banking AI Agents: A Retest Playbook

    Vendor model retirements, alias moves and prompt edits can quietly change how a banking AI agent behaves. Here's what the OCC, DORA, the EU AI Act and the PRA say about change, and a 7-step playbook to retest agents before the switch.

    • model change management
    • llm model updates
    • ai agents
    • banking

    Read

  3. 13 min read

    EU AI Act Credit Scoring: What Banks Must Do Before December 2027

    Creditworthiness AI is high-risk under the EU AI Act, with obligations from 2 December 2027. Here's what's in scope (including LLM agents), who does what, what the Omnibus changed, and a 10-point readiness checklist.

    • eu ai act
    • credit scoring
    • creditworthiness assessment
    • high-risk ai

    Read