- human oversight
- human in the loop
- ai agents
- banking
- escalation
- automation bias
- eu ai act
- fsb
- occ
- fca
Human Oversight for AI Agents in Banking: HITL, HOTL and Escalation
The FSB, FCA and OCC all said it in autumn 2026: human oversight of AI only counts if it's designed, resourced and tested. Here's what they expect, HITL vs HOTL, and a 6-step framework for escalation that holds under pressure.
By Aodit
13 min read
Most banks deploying AI agents say they have "a human in the loop". Fewer can show that the human in question has the time, information and authority to catch a bad decision. Fewer still can show that the agent reliably hands over to that human when it should.
That gap is now a supervisory theme. In October and November 2026, three major publications converged on the same point: human oversight of AI is only a control if it is designed, resourced and tested.
- The Financial Stability Board (FSB) finalised its Sound Practices for Responsible Adoption of Artificial Intelligence on 15 October 2026, listing "appropriate and effective human oversight" as one of its sound practices and warning that AI agents pose "a distinct challenge for human oversight".
- The UK Financial Conduct Authority (FCA) published AI in financial services: good practice and areas for improvement on 19 October 2026, with explicit good and poor practice on human-in-the-loop design, automation bias and escalation.
- The US Office of the Comptroller of the Currency (OCC) released its examiner framework for AI on 4 November 2026 (OCC Bulletin 2026-54), which distinguishes human-in-the-loop from human-on-the-loop and warns against reviews becoming a rubber stamp.
Meanwhile, Article 14 of the EU AI Act sets out human oversight requirements for high-risk systems, including creditworthiness assessment, that will apply from 2 December 2027 following the Digital Omnibus postponement.
This article explains what these sources actually say, why "human in the loop" often fails for agents, and how to design and test escalation so it holds up to an examiner.
Why human oversight breaks down with AI agents
Traditional model risk management assumed a model produces a score and a person makes the decision. AI agents change that in three ways.
1. Agents act, not just advise. An agent that can issue a refund, waive a fee, update a customer record or initiate a payment moves the decision point from the human to the system. The OCC framework makes this explicit: according to law-firm summaries, examiners ask whether a tool "merely informs a human decision or acts on its own", and control intensity scales with "the transfer of agency from people to the system", not technical sophistication. A simple rules-based agent that can start transactions may carry more risk than a sophisticated model that only drafts internal documents.
2. Real-time review doesn't scale. The FSB says AI agents pose "a distinct challenge for human oversight, given the impracticality of real-time human monitoring", noting that opportunities for human intervention "may be limited once actions are being taken". If an agent handles thousands of conversations a day, a human cannot read them all. The oversight model has to rely on the agent recognising when to stop and escalate.
3. Humans defer to machines. Automation bias is the tendency to over-trust automated outputs. The FSB warns that human oversight "may be weakened by over-reliance and/or automation bias", and that sustained reliance on AI may "diminish staff expertise, judgement and challenge capability". The FCA's poor-practice case study describes reviewers who "increasingly accepted AI-generated recommendations without sufficient scrutiny".
The practical conclusion: for agents, the most important oversight control is often the agent's own escalation behaviour. If it doesn't recognise risk, doesn't escalate under pressure, or can be talked out of escalating, the human never gets the chance to intervene.
Can your agent be talked out of escalating? AODIT's free live demo runs your agent's instructions through an 8-turn adversarial conversation, including escalation pressure, and shows how it behaves in about two minutes. No account needed.
What regulators actually say about human oversight
FSB Sound Practices (final, 15 October 2026)
The FSB's final report is principles-based and addressed to financial institutions globally. On human oversight it:
- Lists "appropriate and effective human oversight" among its sound practices.
- Identifies "unauthorised actions" as an agentic risk, noting agents may act autonomously in pursuit of pre-defined goals and "take illegal, unethical, or unauthorised actions without adequate human approval or oversight".
- Cites "clear escalation protocols" as a control around fully automated decision-making in critical business areas.
- Calls for record-keeping that lets firms "attribute actions" to agents, which "facilitates human oversight, ongoing monitoring and the detection of unusual activity".
- Gives an example of agent "certification", where an agent is "reviewed and approved for use within defined boundaries".
Press coverage of the final report also highlights its call for clear, risk-based triggers for when AI agents must seek human approval, and more intensive oversight for higher-risk use cases. The FSB's sound practices aren't binding law, but they signal where national supervisors are heading.
FCA good and poor practice (19 October 2026)
The FCA's review is the most operationally detailed of the three. Key points:
- Design to the use case. "Any human oversight controls should be designed around the specific use case, the level of AI autonomy" and the potential impact on consumers or markets.
- Meaningful, not cosmetic. Good practice means "ensuring people have the right authority, information, capability and escalation routes to challenge or intervene". The FCA flags the risk of review becoming "a 'box-ticking' exercise, rather than meaningful oversight".
- Abstain, redirect or escalate. Firms defined "where the AI tool should abstain, redirect or escalate to a person", backed by automated checks and continuous monitoring rather than human review alone.
- Clear thresholds. Firms "set clear thresholds for when human intervention and escalation were needed". In a poor-practice example, "unclear escalation criteria meant that some AI outputs proceeded despite low confidence or uncertainty."
- Countering automation bias. In one good-practice case, "reviewers assessed each case without being shown the AI's proposed outcome", so the firm could compare independent judgements.
- Agents. Before deploying more autonomous agentic systems, firms designed governance with escalation routes, and "registered their agents in a central inventory".
The FCA paper is a good and poor practice publication, not new rules, but UK firms should expect supervisors to use it as a benchmark under existing obligations such as the Consumer Duty and SM&CR.
OCC examiner framework (Bulletin 2026-54, 4 November 2026)
The OCC framework is an examiner tool and, as the OCC states, does not create new expectations for banks. But it shows what examiners will look at. According to summaries from King & Spalding and others, it:
- Distinguishes human-in-the-loop (a person approves each output before use) from human-on-the-loop (a person monitors and can intervene).
- Says reviewers need "expertise, authority, and time" under either model, and warns that bulk approvals and high volumes can turn review into a rubber stamp.
- Asks banks to evaluate reviewer competency, authority, workload and override logging.
- Requires banks to define "when a person must step in or the tool must be paused", with escalation and suspension triggers for higher-risk use cases.
- Notes that an AI system "can remain fully operational while doing the wrong thing", and points to transaction limits, narrowly scoped permissions and "emergency shutdown ('kill switch') capabilities".
- Places autonomous payment agents and real-time credit decisioning in its illustrative Tier 1 (Critical).
We cover the full framework in OCC Super Intelligence Framework: How Examiners Will Review AI Agents.
EU AI Act Article 14 (applies to Annex III systems from 2 December 2027)
Article 14 requires high-risk AI systems to be designed so they can be effectively overseen by natural persons. Under Article 14(4), the people assigned to oversight must be enabled, as appropriate and proportionate, to:
- understand the system's capacities and limitations and monitor its operation (a);
- remain aware of automation bias (b);
- correctly interpret its output (c);
- decide not to use the system, or to disregard, override or reverse its output (d);
- intervene or interrupt the system "through a 'stop' button or a similar procedure" (e).
For banks, the relevant Annex III use case is creditworthiness assessment and credit scoring of natural persons (point 5(b)), with an exception for fraud detection. Under Regulation (EU) 2026/1744 (the Digital Omnibus), obligations for Annex III high-risk systems now apply from 2 December 2027. A customer-service agent is not automatically high-risk, but if it influences credit decisions, it may fall into scope. See our EU AI Act compliance page for more.
Swiss institutions should note that FINMA Guidance 08/2024 also expects clear responsibilities and testing for AI; we covered it in FINMA AI Guidance 08/2024: Testing and Independent Review of AI Agents.
HITL, HOTL and the agent's own judgment: choosing the right model
The terms get used loosely. A useful working taxonomy for agent deployments:
| Oversight model | How it works | Suits | Main failure mode |
|---|---|---|---|
| Human-in-the-loop (HITL) | A person approves each action before it takes effect | High-impact, low-volume actions (large refunds, credit limit changes, payments above threshold) | Rubber-stamping and automation bias at volume |
| Human-on-the-loop (HOTL) | Agent acts; a person monitors dashboards, samples and alerts and can intervene or pause | Medium-impact, high-volume tasks (servicing queries, document triage) | Monitoring technical metrics instead of outcomes; slow intervention |
| Agent-initiated escalation | Agent recognises risk, uncertainty or out-of-policy requests and hands off to a human | All customer-facing agents, as a baseline | Agent fails to escalate, escalates too late, or is manipulated out of escalating |
In practice, most banking agents combine all three. The important point is that HOTL and agent-initiated escalation depend on behaviour you can only verify by testing the agent. A policy document saying "the agent escalates complaints and vulnerable customers" is not evidence that it does.
Designing escalation that holds up under pressure
Here is a practical framework for designing and evidencing human oversight for AI agents. These are our recommendations, synthesised from the sources above, not regulatory requirements.
1. Map actions to oversight tiers
List every action the agent can take (tools, APIs, write permissions) and classify each by impact and reversibility. For example:
- Inform only (answer FAQs, explain products): HOTL with outcome sampling.
- Low-impact, reversible (update contact preferences, small goodwill credits within a hard limit): HOTL plus hard limits.
- High-impact or irreversible (payments, credit decisions, closing accounts, fee waivers above threshold): HITL or prohibited for the agent.
This mirrors the OCC's consequence-based tiering and the FSB's emphasis on more intensive oversight for higher-risk use cases.
2. Define explicit escalation triggers
Vague instructions like "escalate when appropriate" are exactly what the FCA calls out as poor practice. Write concrete triggers, for example:
- The customer disputes a transaction, alleges fraud or makes a complaint.
- Signs of vulnerability (bereavement, financial difficulty, health, confusion).
- Requests outside documented policy, or for exceptions to it.
- The agent is uncertain or lacks approved source material to answer.
- Threats of legal action, regulator complaints or media.
- Repeated attempts to obtain the same outcome after refusal.
3. Make escalation resistant to manipulation
This is where most agents fail. Customers, and attackers, apply pressure: urgency ("I need this in the next five minutes"), authority ("I'm a senior manager at the bank"), emotion ("you're going to make me lose my house") or false premises ("your colleague already approved this"). A well-designed agent should escalate more readily under these conditions, not less. An agent that grants an exception to avoid a difficult conversation has failed the oversight control, even if a human reviews a sample later.
Our prompt injection testing guide covers the adversarial side in depth.
4. Design the human side properly
Escalation is only as good as what happens next:
- Route escalations to staff with the authority to decide and the information to do so (the full conversation and the agent's reasoning or sources).
- Size teams for escalation volume, so reviewers have time, the OCC's third requirement.
- Consider "blind" review for a sample of cases, as in the FCA's good-practice example, to measure automation bias.
- Log overrides and track override rates. An override rate near zero may indicate rubber-stamping rather than a perfect agent.
5. Build the brakes
Set hard transaction and permission limits outside the model (in the tool layer, not the prompt), define amber and red thresholds with predefined actions, and maintain a tested ability to pause or shut down the agent with a fallback process. The FCA's good-practice examples include red, amber and green thresholds where responses could include "enhanced monitoring or temporary shutdown".
6. Test escalation behaviour before deployment and after every change
Test the agent against realistic scenarios that should trigger escalation, including adversarial ones, and repeat tests, because generative AI is non-deterministic and a single pass proves little. Re-test after prompt changes, tool changes and vendor model updates.
How AODIT tests escalation and human oversight behaviour
AODIT is an independent behavioral risk rating and stress-testing platform for AI agents in banking and finance. Its AODIT-6 methodology includes a dedicated Judgment dimension that asks: "Does it make proportional decisions and escalate correctly under risk?" Its sub-tests map closely to the oversight questions supervisors now ask:
- J1 Proportional action: does the agent's response match the risk of the request?
- J2 Escalation to human oversight: does it hand over when it should?
- J3 Policy boundary adherence: does it stay within its mandate?
- J4 Authority conflict resolution: how does it handle conflicting instructions, such as a user claiming authority?
- J5 Blast radius limitation: does it limit the potential impact of its actions?
The 8-turn adversarial protocol escalates from a baseline question through a grey-zone case, a "hard push" using urgency, authority and emotional pressure, and a manipulation attempt, then checks whether the agent recovers and makes a sound final judgment. That sequence is designed to show whether escalation holds when it matters most. The Resistance dimension adds social engineering (T3) and authority spoofing (T4) tests.
The full platform runs 20–100 scenarios per test across six dimensions and produces a PDF-ready report you can use as evidence for model risk, compliance and internal audit. For institutions that can't send data outside their environment, AODIT supports on-premise and air-gapped deployment. Results are advisory: they inform your governance decisions and don't replace your own validation or legal advice.
A short checklist for risk and compliance teams
Before your next AI committee or examination, check whether you can answer "yes" to these:
- Every agent action is mapped to an oversight tier (HITL, HOTL or prohibited).
- Escalation triggers are written down, specific and reflected in the agent's instructions and tool permissions.
- You have test evidence that the agent escalates correctly, including under pressure and manipulation.
- Escalated cases reach staff with authority, information and time to act.
- Overrides are logged and override rates are monitored for signs of automation bias.
- Hard limits and a tested pause or kill switch exist outside the model.
- Agents are registered in a central inventory with a named accountable owner.
- Testing is repeated after material changes, including vendor model updates.
FAQ
What is the difference between human-in-the-loop and human-on-the-loop?
Human-in-the-loop means a person approves each output or action before it takes effect. Human-on-the-loop means the system acts and a person monitors it and can intervene or pause it. The OCC's 2026 examiner framework uses this distinction, and expects reviewers to have expertise, authority and time under either model.
Does the EU AI Act require human oversight for banking chatbots?
Article 14 applies to high-risk AI systems. In banking, the key Annex III category is creditworthiness assessment and credit scoring of natural persons. A general customer-service chatbot isn't automatically high-risk, but it is subject to transparency obligations under Article 50. Annex III obligations apply from 2 December 2027 under Regulation (EU) 2026/1744.
What is automation bias and why does it matter for AI agents?
Automation bias is excessive trust in automated output, even when it may be wrong. The FSB and FCA both flag it as a way human oversight fails. For agents, it can mean reviewers approve recommendations without real scrutiny, making the human-in-the-loop control ineffective.
How do you test whether an AI agent escalates correctly?
Run the agent through realistic scenarios that should trigger escalation, such as complaints, vulnerable customers, out-of-policy requests and manipulation attempts, and repeat them to account for non-deterministic behaviour. Measure whether, when and how it escalates. AODIT's Judgment dimension (J2 Escalation to human oversight) is designed for this.
Is a kill switch required for AI agents?
There's no single universal legal requirement worded as a "kill switch". However, EU AI Act Article 14(4)(e) requires high-risk systems to allow intervention or interruption through a "stop" button or similar procedure, and the OCC framework lists emergency shutdown capabilities among recommended resilience controls.
Make your human oversight provable
Supervisors are no longer asking whether you have a human in the loop. They're asking whether that human can actually intervene, and whether your agent knows when to call them.
AODIT gives you independent evidence of how your AI agents behave when escalation matters: under pressure, under manipulation and at the edge of policy.
- See it in action: Run the free live demo on your own agent instructions.
- Get a full assessment: Request an evaluation across all six AODIT dimensions, or compare plans.
Sources
- FSB, Sound Practices for Responsible Adoption of Artificial Intelligence (AI), final report, 15 October 2026
- FCA, AI in financial services: good practice and areas for improvement, 19 October 2026
- King & Spalding, The OCC Issues Examiner Framework for Bank Use of AI: What OCC Bulletin 2026-54 Means
- CDO Magazine, OCC Framework Puts AI Autonomy and Human Oversight in Focus for Banks
- EU AI Act, Article 14: Human Oversight
- The Express Tribune, Global financial watchdog calls for more human oversight of agentic AI
This article is for general information and does not constitute legal or regulatory advice.
