Identifying, quantifying and mapping AI risk
In this lesson you will:
- Use an AI risk taxonomy to identify risks systematically
- Write AI risk scenarios that connect threats to business impact
- Rate risks qualitatively and estimate them quantitatively
- Map AI risks to frameworks and an enterprise risk register
AI risk management follows the same cycle as any other risk discipline — identify, assess, treat, monitor — but the sources of risk are new and the impacts can reach people outside the organisation. This lesson covers the first two steps.
1. An AI risk taxonomy
A taxonomy stops you from looking only where you are comfortable. Security teams tend to see attacks; data scientists see accuracy; legal sees compliance. A shared taxonomy brings them together.
| Category | Examples |
|---|---|
| Security | Prompt injection, data poisoning, model theft, agent misuse, supply-chain compromise |
| Privacy | Training-data memorisation, re-identification, unlawful processing, cross-border transfer |
| Performance and safety | Hallucination, drift, unsafe recommendations, brittle behaviour on unusual inputs |
| Fairness and rights | Discriminatory outcomes, lack of contestability, opaque decisions |
| Legal and compliance | Breach of AI, data-protection or sector rules; IP and copyright claims; contractual breach |
| Operational and strategic | Vendor dependency, cost overruns, outages, reputational damage, skills gaps |
2. Writing AI risk scenarios
A risk scenario describes how something could go wrong, to what, and with what consequence. Vague risks ("AI could be hacked") cannot be rated or treated.
A good scenario names:
- Threat source — external attacker, malicious insider, careless user, vendor, the model itself (for non-malicious failures).
- Vector or cause — injected instructions in a document, poisoned training data, a silent provider update.
- AI asset affected — the model, its data, the retrieval store, an agent's tools.
- Event — data disclosed, wrong decision made, unauthorised action taken, service unavailable.
- Impact — financial loss, regulatory penalty, harm to individuals, reputational damage.
Example: "An external attacker plants instructions in a supplier invoice; the accounts-payable agent reads it and changes the supplier's bank details; payments are diverted, causing direct financial loss and a reportable incident."
3. Qualitative rating
Most organisations rate likelihood and impact on a 1–5 scale and plot risks on a heat map. For AI, define impact scales that include harm to people — not only money — and likelihood scales that account for the system's exposure (internal vs public, number of users) and autonomy.
Heat maps are quick and easy to communicate, but they compress uncertainty and invite arguments about whether something is a "3" or a "4". For the most significant risks, quantify.
4. Quantitative estimation
Quantification expresses risk in money and probability, so it can be compared with the cost of controls. A widely used structure (popularised by the FAIR model) breaks risk into:
- Loss event frequency (LEF) — how often the loss event is expected per year. It depends on how often threats act and how often they succeed.
- Loss magnitude (LM) — how much a single event costs: response, replacement, fines and judgements, lost revenue, reputational damage.
- Annualised loss ≈ LEF × LM, usually expressed as a range or a distribution rather than one number.
AI-specific inputs make estimation harder: limited incident history, fast-changing threat techniques and non-deterministic behaviour. Use ranges, calibrated expert estimates and red-team results as evidence of control strength.
Worked example — quantifying a chatbot data-leak scenario
Scenario: an attacker uses prompt injection to make a customer-service assistant reveal other customers' order details.
| Factor | Estimate | Basis |
|---|---|---|
| Attempts per year | ~40 | Public-facing bot; injection attempts seen in logs |
| Probability an attempt succeeds | 5% | Red-team success rate after current guardrails |
| Loss event frequency | ~2 per year | 40 × 5% |
| Records exposed per event | 50–500 | Size of retrievable context |
| Cost per event | ₹15 lakh – ₹1.2 crore | Investigation, notification, customer handling, regulatory exposure |
| Annualised loss (range) | ₹30 lakh – ₹2.4 crore | LEF × LM |
A proposed control — permission-aware retrieval so the bot can only see the current customer's records — is estimated to cut records exposed per event to fewer than 5 and cost per event below ₹5 lakh. Annual cost of the control: ₹25 lakh. The comparison makes the business decision straightforward.
5. Mapping and the risk register
AI risks belong in the enterprise risk register, not a separate spreadsheet that the risk committee never sees. Record for each:
- Scenario description and owner (the business owner of the AI system).
- Linked AI systems (inventory IDs) and assets.
- Inherent rating, existing controls, residual rating.
- Treatment decision and actions.
- Mapping to frameworks — for example NIST AI RMF function, EU AI Act requirement, OWASP LLM Top 10 or MITRE ATLAS technique — so the same risk can be reported to different audiences.
- Key risk indicators (KRIs) and reassessment triggers.
🔐 Security lens
Red-team results are the best evidence you have for the "probability of success" part of likelihood. Feed them into risk ratings explicitly — and rerun them when the model, prompts or guardrails change.
Key takeaways
- Use a shared taxonomy — security, privacy, performance and safety, fairness and rights, legal, operational — so no category is missed.
- Write scenarios with threat source, vector, AI asset, event and impact.
- Heat maps communicate; quantification (LEF × LM, as ranges) supports investment decisions.
- Put AI risks in the enterprise register, owned by the business, mapped to frameworks and linked to the inventory.
Self-check
- Rewrite "AI could leak data" as a complete risk scenario.
- If an AI fraud model is attacked 200 times a year and 2% of attempts succeed, what is the loss event frequency?
- Why should AI risks be in the enterprise register rather than a separate AI risk log?