Ready reckoner
Design principles
The model is untrusted. Zero Trust (verify every call, no trust by location) · defence in depth (no single guardrail) · privacy by design · secure defaults (no tools, read-only scopes, egress off, logging on) · least privilege · fail safe.
Trade-offs to document: latency vs security · explainability vs accuracy · cost vs capability · autonomy vs control · context vs exposure.
| Option | Control | Responsibility |
|---|---|---|
| SaaS AI | Lowest | Vendor-heavy; due diligence |
| Hosted API | Medium | You build guardrails; data leaves boundary |
| Managed in your tenancy | Medium-high | Residency in your cloud |
| Self-hosted open model | High | You patch, scale, secure GPUs, tune safety |
Architecture threats
Client (injection) · orchestrator (policy bypass) · retrieval (poisoning, over-broad) · inference (extraction, retention) · tools (excessive agency) · memory (poisoning) · logs (accumulation). Tools: narrow, typed, server-side authorisation of the user, validated parameters, read ≠ write, no secrets in context, results are untrusted.
MCP and A2A
- MCP: host → client → server (tools, resources, prompts). Remote: OAuth 2.1, audience-bound tokens, no token passthrough. Local servers = user's privileges.
- Risks: tool poisoning · rug pull · shadowing · over-broad scopes · confused deputy. Keep an approved, pinned server registry.
- A2A: Agent Card (identity, skills, auth). Verify identity (mTLS, signed tokens); delegate minimum, time-limited scope; validate results; record chain; circuit breakers.
Infrastructure
Harden notebooks, training clusters, model servers, K8s, GPU nodes, control plane. Sandbox agent code: no creds, no/allow-listed network, limits, destroy after use. Availability: rate limits, compute throttling, caps, agent limits, budget caps, bounded autoscaling. GPUs: dedicated or partitioned with separate memory. Physical: power, cooling, racks, fire suppression.
Guardrails
| Layer | Examples |
|---|---|
| Input | Normalise, length caps, injection detection, topic & PII filters |
| Context | Mark untrusted data, spotlighting, instruction hierarchy, minimise |
| Model | Safety tuning |
| Output | Content & DLP filters, grounding, schema, safe rendering |
| Action (harness) | Policy engine, allow-lists, approvals, budgets, plan review, dual-model, kill switch |
Testing: functional (many phrasings, repeated runs) · performance & load (guardrails on) · stress (past limits) · adversarial · regression. Every failure path fails safe.
Identity
Types: humans · workloads · agents (own ID + owner) · services · tools/connectors. Anti-patterns: agent on a person's account, shared service accounts, copied keys. Non-repudiation: per-agent ID + token exchange / on-behalf-of (subject = user, actor = agent) + tamper-evident logs. M2M: client credentials · token exchange · mTLS · workload identity · proof-of-possession · static keys (last resort, vaulted). SSO everywhere; phishing-resistant MFA for AI admins; federation for partners. Secrets: vault, dynamic, scanned, never in prompts.
Authorisation
PoLP: task-scoped, time-bound, read ≠ write. SoD: propose ≠ approve; write code ≠ deploy. RBAC (roles) · ABAC (attributes) · PBAC (policy as code per call — best for harness) · JIT (temporary elevation). Excessive agency = excessive functionality + permissions + autonomy. Over-delegation → narrowed scopes, capped depth. Lateral movement → per-agent IDs, segmentation, no shared creds. Lifecycle: inventory · provision · deprovision (orphans!) · access reviews · ISPM · audit token use, rate-limit hits, timeouts.
Exam-day reflexes
- "Only defence is the system prompt" → violates defence in depth.
- "Tool description contains hidden instructions" → tool poisoning.
- "Definitions changed after approval" → rug pull; pin + re-approve.
- "Agent proposes and approves payment" → SoD.
- "Who did it — Priya or her agent?" → per-agent identity + on-behalf-of tokens.