AI Threat Modelling, Red Teaming & Assurance, 30 min read

Threat modelling AI systems: STRIDE, PASTA and MAESTRO

In this lesson you will:

  • Apply STRIDE to AI/ML pipelines and applications
  • Describe the PASTA process and when to use it
  • Use MAESTRO's layered view to threat-model multi-agent systems
  • Profile AI threat actors, objectives and insider threats, human and non-human
  • Use AI threat intelligence sources

Threat modelling asks four questions: What are we building? What can go wrong? What are we going to do about it? Did we do a good job? The questions don't change for AI — but the answers to the second one do.

1. STRIDE for AI

STRIDE classifies threats by the security property they violate. Applied to AI:

STRIDE categories with AI-specific examples.
Figure 7.1 STRIDE with AI examples. Walk each component and data flow through all six categories.
Category Property AI examples
Spoofing Authentication Impersonating an agent in A2A; fake MCP server; deepfake voice to a helpdesk agent
Tampering Integrity Poisoned training data; modified weights; edited prompts or retrieval documents
Repudiation Non-repudiation Agent actions not attributable to user or agent; missing tool-call logs
Information disclosure Confidentiality Prompt injection exfiltrating data; training-data extraction; system prompt leakage; model extraction
Denial of service Availability Token exhaustion; sponge inputs; runaway agent loops; GPU starvation
Elevation of privilege Authorisation Injection that makes an agent use tools beyond the user's rights; confused deputy

Method: draw a data-flow diagram with trust boundaries (Module 6), then apply STRIDE to each element and flow. Add AI-specific elements explicitly — training pipeline, model registry, vector store, memory, tools.

2. PASTA

PASTA (Process for Attack Simulation and Threat Analysis) is a risk-centric, seven-stage methodology that connects threats to business impact:

  1. Define business objectives.
  2. Define the technical scope.
  3. Decompose the application.
  4. Analyse threats (using threat intelligence).
  5. Analyse vulnerabilities and weaknesses.
  6. Model and simulate attacks.
  7. Analyse risk and impact, and recommend countermeasures.

PASTA suits high-value AI systems where you want threat modelling tied to business risk and to attack simulation — for example, a customer-facing agent that moves money. It is heavier than STRIDE; many teams use STRIDE for breadth and PASTA for the most critical systems.

3. MAESTRO for agentic and multi-agent systems

MAESTRO (Multi-Agent Environment, Security, Threat, Risk, and Outcome), published by the Cloud Security Alliance, is designed for agentic AI. It models an agentic system as seven layers and asks what can go wrong at each layer and between layers.

MAESTRO seven-layer reference architecture with example threats.
Figure 7.2 MAESTRO's seven layers with example threats. Cross-layer threats — such as poisoned data driving agent actions — are often the most serious.
Layer Example threats
1. Foundation models Backdoors, jailbreaks, model extraction
2. Data operations Poisoned RAG data, embedding manipulation
3. Agent frameworks Tool misuse, orchestration flaws, unsafe plugins
4. Deployment and infrastructure Container escape, resource hijacking
5. Evaluation and observability Manipulated metrics, blind spots in monitoring
6. Security and compliance Policy bypass, privilege escalation, audit gaps
7. Agent ecosystem Agent impersonation, malicious marketplace agents, cascading failures

4. AI threat actors and objectives

Actor Typical objectives Example
Cybercriminals Fraud, data theft, extortion, resale of access Inject instructions to divert payments; steal customer data via an assistant
Nation-state actors Espionage, disruption, influence Steal model weights; poison public datasets; generate disinformation
Competitors IP theft Model extraction; distillation of outputs
Hacktivists and researchers Exposure, embarrassment, publicity Public jailbreaks of a brand's chatbot
Fraudsters and abusers Free compute, policy evasion Account farming to consume quota; generating prohibited content
Malicious insiders Sabotage, theft, fraud Label tampering; downloading weights before leaving
Negligent insiders — (unintended harm) Pasting secrets into chatbots; over-permissive agent set-up
Non-human insiders — (compromised or misaligned agents) An agent with legitimate access manipulated into harmful actions

Non-human insider threats are new: an agent with legitimate, internal access can be turned by injected content into an insider acting against you. Insider-threat programmes should include agent identities and behaviour analytics.

5. AI threat intelligence

Sources include MITRE ATLAS (adversary tactics and techniques against AI, with case studies), the OWASP Top 10 for LLM Applications and related OWASP agentic-security guidance, AI incident databases, model-provider security advisories, vulnerability databases for AI frameworks, and academic research. Feed them into threat models and red-team scenarios.

Worked example — STRIDE on an HR policy assistant

Element / flow STRIDE finding Mitigation
Employee → assistant S: no re-authentication for sensitive questions SSO session + step-up for personal-data queries
HR documents → vector index T: any HR staff member can upload documents Upload restricted to policy owners; content review
Assistant → leave-request tool E: tool uses a service account that can edit anyone's leave Token exchange; tool checks the requesting employee
Assistant answers I: retrieval includes investigation files Index only published policies; ACL filtering
Tool calls R: no logs of who requested changes Log user, agent, tool, parameters
Public holidays query spike D: no per-user rate limit Rate limit + caching of common answers

🔐 Security lens

Threat-model the workflow, not just the model. Most serious AI findings come from what surrounds the model — retrieval, tools, identities and memory — and from flows across trust boundaries.

Key takeaways

  • STRIDE gives breadth: spoofing, tampering, repudiation, information disclosure, denial of service, elevation of privilege — each with AI-specific forms.
  • PASTA ties threats to business objectives and attack simulation in seven stages; use it for the most critical systems.
  • MAESTRO models agentic systems in seven layers and emphasises cross-layer threats.
  • Threat actors range from criminals and states to insiders — including non-human insiders: manipulated agents with legitimate access.
  • Use MITRE ATLAS, OWASP, incident databases and advisories as AI threat intelligence.

Self-check

  1. Which STRIDE category does training-data poisoning fall under?
  2. When would you choose PASTA over STRIDE?
  3. What is a non-human insider threat?