Ready reckoner
Traditional vs AI: seven differences
| Aspect | Traditional | AI | So… |
|---|---|---|---|
| Behaviour | Deterministic | Probabilistic | Evaluate many inputs; assume errors |
| Instructions vs data | Separated | One context | Any untrusted text = possible instruction |
| Code | Code | Code + data + weights | Datasets and pipelines are attack surface |
| Assets | Code, data, secrets | + weights, datasets, prompts, embeddings, memory | Protect and inventory them |
| Supply chain | Libraries, images | + base models, datasets, adapters, MCP servers, peer agents | Pin, scan, sign, allow-list |
| Who acts | Users, services | + agents with delegated authority | Least agency; confused deputy |
| Validation | Tests pass/fail | Evals, red teaming, monitoring | Re-test on model/prompt/tool change |
Golden rule: treat the model as untrusted; enforce security outside it. Prompts and guardrail models are layers.
Lifecycle threats
Data (poisoning, PII) → Train (backdoors) → Evaluate (weak evals) → Package (model swap, unsafe pickle) → Deploy (injection, evasion, theft) → Operate (drift, feedback poisoning).
STRIDE for AI
S deepfakes, agent/tool impersonation · T poisoning, prompt injection, memory poisoning · R untraceable agent actions · I extraction, leakage, RAG oversharing, hidden context · D unbounded consumption, loops · E excessive agency, tool misuse, confused deputy.
OWASP LLM Top 10 (2026)
01 Prompt Injection · 02 Sensitive Info Disclosure · 03 Excessive Agency · 04 Supply Chain · 05 Data & Model Poisoning · 06 Unbounded Consumption · 07 Misinformation · 08 Hidden Context Exposure · 09 Vector & Embedding Weaknesses · 10 Improper Output Handling. (2025 numbering differs — cite the edition.)
OWASP Agentic Top 10 (ASI, Dec 2025)
01 Goal Hijack · 02 Tool Misuse · 03 Identity & Privilege Abuse · 04 Agentic Supply Chain · 05 Unexpected Code Execution · 06 Memory & Context Poisoning · 07 Insecure Inter-Agent Comms · 08 Cascading Failures · 09 Human-Agent Trust Exploitation · 10 Rogue Agents. Principles: least agency, strong observability.
Other frameworks
MITRE ATLAS — adversary tactics/techniques + case studies (attack paths, red teams). NIST AI 100-2 E2025 — evasion, poisoning, privacy (incl. extraction, membership inference), GenAI injection/misuse. MAESTRO (CSA) — 7 layers: foundation models · data ops · agent frameworks · deployment/infra · evaluation/observability · security & compliance · ecosystem; look for cross-layer threats.
Three examples
| ML model | RAG assistant | Agent | |
|---|---|---|---|
| Risk enters via | Data, labels, feedback, base model, evasion | Retrieved content, prompts, output | Any text read + permissions |
| Key controls | Review before learning; pinned/scanned/signed models; liveness; hide scores; attempt limits | Curated corpus; permission-aware retrieval; session-bound tools; safe rendering; grounding | Tool/permission table; parameter-free tools; caps; delegated identity; reader/actor split; approvals with original evidence; MCP/A2A auth; circuit breakers; traces |
Agent question: if an attacker controlled any text this agent reads, what is the worst it could do with its permissions?