Impact assessments, blast radius and risk tiering
In this lesson you will:
- Conduct an AI impact assessment covering algorithmic risk and autonomous actions
- Estimate the blast radius of an AI failure or compromise
- Assign use-case risk tiers based on human impact and technical complexity
Not every AI system deserves the same scrutiny. A spell-checker and a loan-decision engine both use AI, but only one can change someone's life. Impact assessment and tiering make sure effort goes where harm is possible.
1. AI impact assessments
An AI impact assessment (AIIA) examines the potential effects of an AI system on individuals, groups, the organisation and society, before deployment and at major changes. It complements — and often shares inputs with — a data protection impact assessment (DPIA) and, under the EU AI Act, a deployer's fundamental rights impact assessment for certain high-risk uses. ISO/IEC 42005 provides guidance on AI system impact assessment.
Typical structure:
- System description — purpose, intended and foreseeable misuse, users, affected people.
- Decision context — what decisions the AI informs or makes, and their significance.
- Algorithmic risk — accuracy across groups, bias, explainability, robustness.
- Autonomy and actions — what the system can do without a human, and how reversible those actions are.
- Data — personal and sensitive data, provenance, consent and lawful basis.
- Security — threat model summary and key attack paths.
- Stakeholder impact — benefits and harms for each affected group; vulnerable groups.
- Mitigations and residual risk — controls, oversight, monitoring, sign-off.
2. Blast radius
Blast radius is how far damage can spread if the AI system fails or is compromised. It is driven less by the model and more by what the system is connected to and allowed to do.
Questions that size the blast radius:
- Data reach — what data can it read? Whose data?
- Action reach — what can it change, send, buy, delete or deploy?
- Autonomy — does a human approve actions, and how fast does it act?
- Scale — how many users, decisions or transactions per hour?
- Propagation — can its output feed other systems or agents automatically?
- Reversibility — can effects be undone?
Reducing blast radius is often cheaper than reducing likelihood: scope permissions narrowly, add approval gates for irreversible actions, rate-limit, and isolate tenants.
3. Use-case risk tiers
Tiers turn assessment results into a consistent level of control. A common approach plots human impact against technical complexity and autonomy.
| Tier | Typical characteristics | Required controls (example) |
|---|---|---|
| Tier 1 — Low | Internal productivity, human reviews all output, no personal decisions | Register, acceptable-use rules, approved tool |
| Tier 2 — Medium | Customer-facing content, moderate data access, limited actions | + security review, testing, monitoring, disclosure |
| Tier 3 — High | Significant decisions about people, autonomous actions, sensitive data, regulated uses | + full impact assessment, bias testing, red teaming, HITL, committee approval, periodic audit |
| Prohibited | Uses banned by law or by the organisation's ethics policy | Not permitted |
Regulatory classification (for example EU AI Act high-risk) sets a floor: an EU high-risk use can never be below your top internal tier.
Worked example — tiering three use cases
| Use case | Human impact | Complexity / autonomy | Blast radius | Tier |
|---|---|---|---|---|
| Meeting-summary assistant for internal staff | Low — staff check summaries | Low — no actions | Meeting content of attendees | Tier 1 |
| Website chatbot answering policy questions with RAG | Medium — customers act on answers | Medium — retrieval, no actions | Public; all indexed documents | Tier 2 |
| Agent that approves small insurance claims automatically | High — payment decisions about people | High — autonomous payments | All claims up to the cap; payment system | Tier 3 |
For the claims agent, the assessment recommends cutting blast radius before go-live: a payment cap per claim and per day, HITL for rejections, and read-only access to policy data.
🔐 Security lens
Ask for the agent's tool list and permission scopes during the impact assessment, not after the build. Blast radius is decided by those scopes.
Key takeaways
- AI impact assessments cover purpose, decisions, algorithmic risk, autonomy, data, security, stakeholders and mitigations.
- Blast radius depends on data reach, action reach, autonomy, scale, propagation and reversibility.
- Tiers link assessment results to required controls; legal classification sets the minimum tier.
- Shrinking blast radius is often the most cost-effective treatment.
Self-check
- Name three factors that increase an AI agent's blast radius.
- Why should an EU high-risk use never sit in your lowest internal tier?
- Which section of an impact assessment would record that an agent can issue refunds without approval?