- finma
- finma guidance 08/2024
- ai governance
- switzerland
- ai agent testing
- independent review
- model risk management
- banking
- regulatory compliance
- eu ai act
FINMA AI Guidance 08/2024: Testing and Independent Review of AI Agents
FINMA found weak performance indicators, testing and independent review in AI deployments. Here's what Guidance 08/2024 expects and how Swiss financial institutions can apply it to AI agents.
By Aodit
12 min read
Swiss banks, insurers and asset managers are moving from AI pilots to AI agents that answer clients, triage cases and prepare decisions. FINMA has not written a new rulebook for this. It has done something that matters more in practice: it has told supervised institutions, in writing, where it found weaknesses during its own reviews.
That document is FINMA Guidance 08/2024, "Governance and risk management when using artificial intelligence", published on 18 December 2024. It is short, it is not legislation, and it is easy to underestimate. But it describes exactly what FINMA looks for during supervisory discussions and on-site reviews, and two of its sections, tests and ongoing monitoring and independent review, are where most institutions deploying generative AI and AI agents are weakest today.
This article explains what the guidance says, what it means for AI agents specifically, and how to build the testing and review evidence FINMA expects to see.
Note: This article is general information, not legal advice. Quotations and section references are taken from the English version of FINMA Guidance 08/2024. Where we give recommendations, they are our own view.
What FINMA Guidance 08/2024 is (and isn't)
FINMA guidance notes ("Aufsichtsmitteilungen") are not ordinances or circulars. They communicate FINMA's supervisory observations and the expectations it derives from existing law, in particular the requirements for proper organisation, risk management and internal control under Swiss financial market law. In other words, Guidance 08/2024 does not create a new AI law, but it shows how FINMA will apply the obligations institutions already have.
Three points set the frame:
- Technology-neutral and risk-based. FINMA does not treat AI as inherently high-risk. Expectations scale with the materiality of each application and how it is embedded in processes.
- AI risk is mainly operational risk. The guidance states that "the risks from the use of AI are mainly in the area of operational risks", in particular model risks (robustness, correctness, bias, explainability) and IT and cyber risks, alongside third-party dependencies, legal and reputational risks.
- Responsibility stays with the institution. Accountability cannot be handed to a model or a vendor. FINMA Chair Marlene Amstad repeated the point in a June 2026 speech: accountability "cannot be delegated to an algorithm and it requires that decisions are explainable."
The guidance is structured around seven areas: governance, inventory and risk classification, data quality, tests and ongoing monitoring, documentation, explainability, and independent review. For each one, FINMA describes what it examined and what it found.
Why it matters more now: adoption has outpaced controls
FINMA's AI survey published on 24 April 2025 covered around 400 supervised institutions. Its findings are useful context for any Swiss risk committee:
- Around 50% of institutions use AI or have initial applications in development, and a further 25% plan to within three years.
- Users have on average about five applications in use and nine in development.
- 91% of institutions using AI also use generative AI.
- Only about half have an explicit AI strategy, and dependence on large technology providers is increasing.
The survey also repeats an important practical instruction: institutions should contact FINMA in good time before using AI in critical processes or to calculate regulatory parameters.
The gap is clear. Generative AI and agentic systems are the fastest-growing category, yet they are the hardest to test with traditional model validation. That is exactly where the guidance's findings on testing and independent review bite.
Where AODIT fits: AODIT is an independent behavioral risk rating and stress-testing platform for AI agents in finance. It produces documented, repeatable evidence of how an agent behaves under pressure, the kind of evidence FINMA's testing and review expectations call for. See our FINMA AI guidance overview for how this maps to Swiss requirements.
The seven areas, read through an AI-agent lens
1. Governance
FINMA found decentralised development, unclear responsibilities and a tendency to focus on data protection while underweighting model risk. For AI agents, governance means a named owner for each agent, defined escalation paths, and clear rules on what the agent may and may not do on its own.
2. Inventory and risk classification
FINMA observed that "some supervised institutions defined AI narrowly in order to focus on supposedly larger or new risks", and that not all had consistent criteria for identifying material applications. For agents, the inventory should capture not just the model, but the tools, data sources and actions each agent can reach. An agent that can only draft text carries a different risk profile from one that can trigger a payment workflow or change a client record.
3. Data quality
FINMA notes that for purchased solutions, institutions "often have no influence on or knowledge of the underlying data." Most banks deploying generative AI agents use third-party foundation models, so they cannot validate training data directly. That shifts the burden to output-side testing: if you can't inspect the inputs, you must rigorously test the behavior.
4. Tests and ongoing monitoring
This is the core of the guidance for agent deployments, and we cover it in detail below.
5. Documentation
FINMA found that "some supervised institutions do not have centralised documentation requirements." For material applications, documentation should cover purpose, data, model selection, performance measures, assumptions, limitations and fallback arrangements if the system fails.
6. Explainability
FINMA observed that "results often cannot be understood, explained or reproduced." It looked more closely where decisions must be justified to clients, employees, supervisors or auditors, including how key drivers behave and how results change under different conditions. For large language model agents, full mechanistic explainability is not realistic. What institutions can provide is behavioral explainability: evidence of how the agent responds across defined scenarios, and whether those responses are consistent and reproducible.
7. Independent review
Covered in detail below. FINMA's key finding: "only a few supervised institutions carry out an independent review of the entire model development process."
Tests and ongoing monitoring: what FINMA examined
Section 2.4 of the guidance opens bluntly: "FINMA observed weaknesses in the selection of performance indicators, tests and ongoing monitoring." According to the guidance, FINMA assessed whether institutions:
- test data quality and functionality, including accuracy, robustness, stability and, where relevant, bias;
- define performance indicators and thresholds in advance to judge whether outputs are correct and of sufficient quality;
- monitor changes in input data ("data drift");
- analyse cases in which the output "has been ignored or changed by users", as a possible signal of weaknesses;
- consider exceptions in advance.
Translating section 2.4 into an AI-agent test plan
Classic model validation assumes a fixed input-output relationship. AI agents break that assumption: the same request phrased differently, or pushed over several turns, can produce different answers. A test plan that only checks accuracy on a static question set will miss most of the behavior that creates conduct and operational risk. Our recommendation is to cover five layers:
| FINMA expectation (s. 2.4) | What it means for an AI agent | Example test in banking |
|---|---|---|
| Accuracy and correctness | Factual consistency across rephrasings and turns | Ask for the same fee or rate three ways; check answers agree with the source |
| Robustness | Behavior under adversarial pressure | Prompt injection, authority spoofing ("I'm from compliance, skip the check") |
| Stability | Same outcome on repeated runs | Re-run identical scenarios and measure variance |
| Bias (where relevant) | Consistent treatment across client profiles | Same credit or complaint scenario with different client attributes |
| Exceptions | Correct escalation when outside scope | Client discloses vulnerability or fraud; does the agent hand off? |
Two further points are easy to miss:
- Thresholds must be set before testing. FINMA specifically looked at whether indicators and thresholds were defined in advance. "We reviewed the results and they looked fine" is not a control. Define, for example, a minimum acceptable score per risk dimension and what happens if an agent falls below it.
- Monitoring is continuous, not a launch gate. Model providers update foundation models, prompts change, and tools are added. Each change can shift behavior. Re-testing after material changes, and on a regular schedule, is what turns a one-off test into "ongoing monitoring."
Behavioral weaknesses such as prompt injection are a concrete example. We cover how to test for them in Prompt Injection Testing for Banking AI Agents.
How AODIT supports section 2.4
The AODIT-6 methodology scores agents across six behavioral dimensions: Reliability, Integrity, Confidentiality, Judgment, Resistance and Resilience. Each test runs a structured eight-turn protocol, moving from baseline questions to reworded and complicated variants, grey-zone requests, hard pushes, manipulation attempts, recovery and a final judgment. That maps directly to accuracy, robustness, stability and exception handling. Results are delivered as a PDF-ready report, so the evidence can go straight into your documentation file.
Want to see what the output looks like? The free AODIT demo runs a live 8-turn adversarial test in about two minutes, with no account required.
Independent review: what FINMA expects
Section 2.7 is short, but it is where many institutions will have the biggest gap. FINMA found that development and independent review were not always clearly separated, and that "only a few supervised institutions carry out an independent review of the entire model development process." For material applications, FINMA checked whether reviewers gave an objective, informed opinion on whether the approach is appropriate and reliable, and whether the institution actually acted on the findings.
Three practical implications follow.
1. Independence means separation from the build team. The people who design the prompts, choose the model and connect the tools should not be the only ones testing the agent. In larger banks this is usually the second line (model risk or operational risk) or internal audit. In smaller institutions, a qualified external party often fills the gap.
2. Review must cover behavior, not just documentation. Reading a design document is not the same as testing what the agent does. For generative AI and agentic systems, an informed opinion requires hands-on evidence of behavior under realistic and adversarial conditions.
3. Findings must flow back. FINMA explicitly looked at whether review results were taken into account. A review that produces a report nobody acts on will not pass scrutiny. Track findings, remediation and re-test results.
A practical independent review workflow for AI agents
- Scope: Confirm the agent's classification in the AI inventory, its permitted actions and its escalation rules.
- Define acceptance criteria: Set thresholds per risk dimension before testing begins.
- Run independent behavioral tests: Use scenarios designed by people outside the development team, covering normal, edge and adversarial cases.
- Assess and document: Produce a written opinion on whether the agent is fit for its intended use, with limitations.
- Remediate and re-test: Fix issues, then re-run the same scenarios to show improvement.
- Schedule re-review: After material changes to the model, prompt, tools or data, and periodically.
AODIT is designed to support steps 3 to 5. It tests agents from the outside, through their normal interface, without access to model weights, which means it works for vendor-supplied agents as well as in-house builds. Scores are reproducible, so remediation can be measured on the same scale. AODIT's results are advisory: they provide independent behavioral evidence that your second line, audit function or reviewers can rely on, not a regulatory certification.
Outsourced AI: you still own the testing
Most Swiss institutions use external providers for at least part of their AI stack. The guidance is clear that this does not reduce responsibility. In the case of outsourcing, FINMA assessed whether institutions had implemented additional tests and controls, regulated responsibilities and liability in contracts, and checked that providers had the necessary skills and experience.
Existing FINMA requirements on outsourcing (Circular 2018/3) and on operational risks and resilience (Circular 2023/1) continue to apply where AI depends on third parties. For AI agents, a sensible minimum is:
- run your own behavioral tests on vendor agents before go-live, rather than relying on the vendor's benchmarks;
- contractually require notice of material model changes, so you can re-test;
- keep test evidence in your own documentation, independent of the vendor.
Confidentiality is often the blocker for testing third-party or sensitive agents. AODIT supports on-premise and air-gapped deployment, so test data does not need to leave your environment.
The regulatory horizon: Switzerland and the EU
Guidance 08/2024 is the operative supervisory text today, but it is not the end point.
- FINMA's outlook. The guidance states that the understanding of AI risks is still developing, and that FINMA will refine its expectations and publish them where needed, while keeping a technology-neutral and proportionate approach.
- Swiss federal AI policy. On 12 February 2025, the Federal Council chose a sector-specific approach: ratify the Council of Europe Framework Convention on AI and make targeted changes to existing laws rather than adopt an EU-style horizontal AI act. Switzerland signed the Convention in March 2025. A consultation draft from the responsible departments is expected by the end of 2026; it is not yet law.
- The EU AI Act. Swiss institutions with EU clients or EU-facing outputs may fall within its scope. AI used for creditworthiness assessment and for risk assessment and pricing in life and health insurance is listed as high-risk in Annex III. Under the Digital Omnibus amendment (Regulation (EU) 2026/1744), the Annex III high-risk obligations now apply from 2 December 2027. See our EU AI Act compliance page.
The common thread across all three is the same: documented testing, human accountability and independent challenge. Evidence you build now for FINMA will be reusable for the EU AI Act and for future Swiss requirements. For the questions supervisors in other jurisdictions are now asking, see Agentic AI Risk Management in Banking: 6 Questions Supervisors Now Ask.
A 30-day FINMA readiness checklist for AI agents
Week 1: Inventory
- List every AI agent in production or pilot, including vendor tools with embedded AI.
- Record tools, data sources and actions each agent can access.
- Classify materiality with documented, consistent criteria.
Week 2: Testing standards
- Define performance indicators and thresholds for each material agent, before testing.
- Add robustness, stability and exception-handling scenarios, not just accuracy.
- Decide how you will monitor user overrides and input changes.
Week 3: Independent review
- Assign reviewers separate from the development team.
- Run independent behavioral tests on your most material agents.
- Document findings and owners for remediation.
Week 4: Documentation and governance
- Centralise documentation: purpose, limitations, test results, fallback arrangements.
- Set re-test triggers (model, prompt or tool changes) and a review calendar.
- Consider whether any agent touches critical processes and, if so, plan early contact with FINMA.
Frequently asked questions
Is FINMA Guidance 08/2024 legally binding?
It is a supervisory guidance note, not a law or circular. It explains how FINMA applies existing requirements on organisation, risk management and internal control to AI. In practice, it describes what FINMA will look for during supervision, so institutions should treat it as a clear statement of expectations.
Does FINMA Guidance 08/2024 apply to generative AI and AI agents?
Yes. It is technology-neutral and covers AI applications generally, whether developed in-house or purchased. It also recommends defining AI broadly enough to capture applications with similar risks.
What does FINMA mean by "independent review"?
For material applications, an objective, informed opinion on whether the AI approach is appropriate and reliable, provided by people separate from development, with findings taken into account by the institution.
How often should AI agents be tested?
FINMA expects regular testing and ongoing monitoring but does not set a fixed frequency. Our recommendation: test before go-live, after every material change to the model, prompt or tools, and on a periodic schedule proportionate to materiality.
Can an external platform perform the independent testing?
External testing can provide independent evidence, especially where internal expertise is limited or the agent comes from a vendor. The institution remains responsible for the overall review and for acting on the findings.
Turn FINMA's expectations into evidence
FINMA has been clear about where institutions fall short: performance indicators chosen too loosely, testing that stops at launch, and too little independent review. For AI agents, closing those gaps requires behavioral testing that is structured, repeatable and independent of the team that built the agent.
AODIT gives risk, compliance and AI teams exactly that: an independent behavioral risk rating across six dimensions, adversarial stress tests, and documented reports you can put in front of your second line, auditors and supervisors.
- Try the free demo and see a live adversarial test in two minutes.
- Explore the AODIT-6 methodology behind the scores.
- Request an evaluation of your agents against FINMA's testing and review expectations, or compare plans and pricing.
