AI Risk Management & the AI Supply Chain, 30 min read

Identifying, quantifying and mapping AI risk

In this lesson you will:

  • Use an AI risk taxonomy to identify risks systematically
  • Write AI risk scenarios that connect threats to business impact
  • Rate risks qualitatively and estimate them quantitatively
  • Map AI risks to frameworks and an enterprise risk register

AI risk management follows the same cycle as any other risk discipline — identify, assess, treat, monitor — but the sources of risk are new and the impacts can reach people outside the organisation. This lesson covers the first two steps.

1. An AI risk taxonomy

A taxonomy stops you from looking only where you are comfortable. Security teams tend to see attacks; data scientists see accuracy; legal sees compliance. A shared taxonomy brings them together.

Six categories of AI risk with examples.
Figure 3.1 A practical AI risk taxonomy. Most real scenarios touch more than one category.
Category Examples
Security Prompt injection, data poisoning, model theft, agent misuse, supply-chain compromise
Privacy Training-data memorisation, re-identification, unlawful processing, cross-border transfer
Performance and safety Hallucination, drift, unsafe recommendations, brittle behaviour on unusual inputs
Fairness and rights Discriminatory outcomes, lack of contestability, opaque decisions
Legal and compliance Breach of AI, data-protection or sector rules; IP and copyright claims; contractual breach
Operational and strategic Vendor dependency, cost overruns, outages, reputational damage, skills gaps

2. Writing AI risk scenarios

A risk scenario describes how something could go wrong, to what, and with what consequence. Vague risks ("AI could be hacked") cannot be rated or treated.

Anatomy of an AI risk scenario from threat source to business impact.
Figure 3.2 Anatomy of a risk scenario. Each element is something you can control, measure or insure against.

A good scenario names:

  1. Threat source — external attacker, malicious insider, careless user, vendor, the model itself (for non-malicious failures).
  2. Vector or cause — injected instructions in a document, poisoned training data, a silent provider update.
  3. AI asset affected — the model, its data, the retrieval store, an agent's tools.
  4. Event — data disclosed, wrong decision made, unauthorised action taken, service unavailable.
  5. Impact — financial loss, regulatory penalty, harm to individuals, reputational damage.

Example: "An external attacker plants instructions in a supplier invoice; the accounts-payable agent reads it and changes the supplier's bank details; payments are diverted, causing direct financial loss and a reportable incident."

3. Qualitative rating

Most organisations rate likelihood and impact on a 1–5 scale and plot risks on a heat map. For AI, define impact scales that include harm to people — not only money — and likelihood scales that account for the system's exposure (internal vs public, number of users) and autonomy.

Five-by-five heat map with six AI risks plotted.
Figure 3.3 Six AI risk scenarios on a 5×5 heat map. Agentic and public-facing scenarios cluster towards the top right.

Heat maps are quick and easy to communicate, but they compress uncertainty and invite arguments about whether something is a "3" or a "4". For the most significant risks, quantify.

4. Quantitative estimation

Quantification expresses risk in money and probability, so it can be compared with the cost of controls. A widely used structure (popularised by the FAIR model) breaks risk into:

  • Loss event frequency (LEF) — how often the loss event is expected per year. It depends on how often threats act and how often they succeed.
  • Loss magnitude (LM) — how much a single event costs: response, replacement, fines and judgements, lost revenue, reputational damage.
  • Annualised loss ≈ LEF × LM, usually expressed as a range or a distribution rather than one number.

AI-specific inputs make estimation harder: limited incident history, fast-changing threat techniques and non-deterministic behaviour. Use ranges, calibrated expert estimates and red-team results as evidence of control strength.

Worked example — quantifying a chatbot data-leak scenario

Scenario: an attacker uses prompt injection to make a customer-service assistant reveal other customers' order details.

Factor Estimate Basis
Attempts per year ~40 Public-facing bot; injection attempts seen in logs
Probability an attempt succeeds 5% Red-team success rate after current guardrails
Loss event frequency ~2 per year 40 × 5%
Records exposed per event 50–500 Size of retrievable context
Cost per event ₹15 lakh – ₹1.2 crore Investigation, notification, customer handling, regulatory exposure
Annualised loss (range) ₹30 lakh – ₹2.4 crore LEF × LM

A proposed control — permission-aware retrieval so the bot can only see the current customer's records — is estimated to cut records exposed per event to fewer than 5 and cost per event below ₹5 lakh. Annual cost of the control: ₹25 lakh. The comparison makes the business decision straightforward.

5. Mapping and the risk register

AI risks belong in the enterprise risk register, not a separate spreadsheet that the risk committee never sees. Record for each:

  • Scenario description and owner (the business owner of the AI system).
  • Linked AI systems (inventory IDs) and assets.
  • Inherent rating, existing controls, residual rating.
  • Treatment decision and actions.
  • Mapping to frameworks — for example NIST AI RMF function, EU AI Act requirement, OWASP LLM Top 10 or MITRE ATLAS technique — so the same risk can be reported to different audiences.
  • Key risk indicators (KRIs) and reassessment triggers.

🔐 Security lens

Red-team results are the best evidence you have for the "probability of success" part of likelihood. Feed them into risk ratings explicitly — and rerun them when the model, prompts or guardrails change.

Key takeaways

  • Use a shared taxonomy — security, privacy, performance and safety, fairness and rights, legal, operational — so no category is missed.
  • Write scenarios with threat source, vector, AI asset, event and impact.
  • Heat maps communicate; quantification (LEF × LM, as ranges) supports investment decisions.
  • Put AI risks in the enterprise register, owned by the business, mapped to frameworks and linked to the inventory.

Self-check

  1. Rewrite "AI could leak data" as a complete risk scenario.
  2. If an AI fraud model is attacked 200 times a year and 2% of attempts succeed, what is the loss event frequency?
  3. Why should AI risks be in the enterprise register rather than a separate AI risk log?