Treating AI risk and knowing when to reassess
In this lesson you will:
- Apply mitigate, transfer, avoid and accept to AI risks
- Select controls for each AI component layer
- Define reassessment triggers and key risk indicators for AI systems
Once a risk is assessed, someone has to decide what to do about it. The options are familiar; the controls and the reasons to revisit the decision are new.
1. Treatment options
| Option | Meaning | AI example |
|---|---|---|
| Mitigate (reduce) | Implement controls to reduce likelihood or impact | Add permission-aware retrieval, output filtering, HITL for refunds |
| Transfer (share) | Shift financial consequences to another party | Contractual indemnities from the model provider; cyber insurance that covers AI incidents |
| Avoid | Stop or redesign the activity | Do not let the agent send external email; use a rules engine instead of an LLM for eligibility decisions |
| Accept | Consciously retain the risk within appetite | Accept residual hallucination risk in an internal drafting tool with human review |
Two cautions:
- Transfer moves money, not accountability. If a vendor's model discriminates against your customers, regulators and customers still hold you responsible.
- Acceptance must be explicit, made by someone with authority, recorded with rationale and an expiry date, and revisited when triggers fire.
2. Controls by AI component
Treat the AI system as a stack of components and ask what control fits each layer.
| Layer | Example controls |
|---|---|
| Data | Provenance checks, validation, access control, minimisation, poisoning detection |
| Model | Approved registry, signing and hashing, evaluation gates, adversarial robustness testing |
| Prompt / context | Context isolation, untrusted-content marking, prompt hardening, secrets kept out |
| Application | Input and output filtering, DLP, rate limiting, authentication |
| Agent / tools | Least-privilege scopes, allow-listed tools, approval gates, spend and iteration caps |
| Infrastructure | Isolation, hardening, encryption, monitoring, capacity limits |
| People and process | Training, HITL, change management, incident playbooks |
Control types still apply: preventive (permission scopes), detective (monitoring tool calls), corrective (rollback), deterrent (acceptable-use warnings) and compensating (extra review where a primary control is not possible, such as for a legacy model that cannot be retrained).
3. Reassessment triggers
AI risk ratings decay faster than most. Define events that force a reassessment instead of waiting for the annual cycle.
- Model change — new model version, provider update, fine-tune, new adapter.
- Behavioural or data drift beyond thresholds.
- Vulnerability disclosure — a new jailbreak technique, a CVE in a serving library, a provider security advisory.
- New use or new users — expansion to customers, a new country, a new decision type.
- New data sources or tools — connecting the agent to email, adding a retrieval source.
- Incidents and near misses — internally or at peers using the same model.
- Regulatory change — a new obligation or a changed classification.
- Time — a maximum interval (for example 12 months for Tier 3).
Key risk indicators
KRIs give early warning between reassessments. Useful AI KRIs include: rate of blocked prompt-injection attempts; guardrail trigger rates; human override rate; output-quality scores; drift metrics; number of unregistered AI services detected; mean time to patch AI dependencies; cost per request.
Worked example — treating risks for a GenAI customer assistant
| Risk | Treatment | Detail |
|---|---|---|
| Disclosure of other customers' data via injection | Mitigate | Retrieval filtered to the authenticated customer; output DLP; red-team quarterly |
| Wrong refund promises (hallucination) | Mitigate + avoid | Assistant cannot commit to refunds; it hands off to a human agent |
| Provider outage | Transfer + mitigate | SLA credits in contract; fallback to a smaller hosted model and a static FAQ |
| Residual offensive output despite filters | Accept | Accepted by the Head of Customer Service, within appetite, review in 6 months or on any incident |
Triggers recorded: provider model update, new product line added to retrieval, any confirmed data-exposure incident, override rate above 15%.
🔐 Security lens
Subscribe the AI system owners — not only the security team — to provider security advisories and model-change notices. A silent model update is a reassessment trigger that often goes unnoticed.
Key takeaways
- Mitigate, transfer, avoid, accept — and record acceptance with authority, rationale and expiry.
- Transfer shifts cost, never accountability.
- Map controls to data, model, context, application, agent, infrastructure and people layers.
- Reassess on model change, drift, vulnerabilities, new uses, new data or tools, incidents, regulation and time; monitor KRIs in between.
Self-check
- Give an example of avoiding, rather than mitigating, an AI risk.
- Why does contractual indemnity from a model vendor not remove a deployer's regulatory exposure?
- List three AI-specific key risk indicators.