Ready reckoner
Architecture principles
- The model cannot enforce security; the architecture must. Zero Trust on every call, defense in depth, privacy by design, secure defaults.
- Main control points: AI gateway and orchestrator/harness.
- Threat-model inputs, retrieval, model, endpoints, tools/agents, memory, pipelines, outputs. Watch confused deputies.
- Tools/MCP/A2A: least privilege, allow-lists, pinned definitions, validated calls and results, no token passthrough, human approval for high impact.
Infrastructure
- ML tooling (notebooks, trackers) is the most common starting point: authenticate, never expose publicly.
- Isolation ladder: process → container → hardened runtime → micro-VM → VM → dedicated → confidential computing. Agent-generated code = least trusted → sandbox, no network, no secrets, destroyed after each task.
- Exhaustion controls: rate and token limits, quotas, timeouts, capped autoscaling. Clear GPU memory between jobs.
Pipelines (MLOps / LLMOps)
- Prompts, indexes, guardrail configs and tool definitions are release artifacts. Whoever controls the pipeline controls production.
- Ephemeral runners, federated short-lived credentials, pinned inputs, signed minimal images, protected pipeline definitions.
- Data zones: raw → validated → curated (only the pipeline writes to curated).
- Gates are enforceable via signed attestations + policy as code + separation of duties.
Guardrails
| Layer | Example |
|---|---|
| Input | Injection/jailbreak classifiers, size limits |
| Context | Separate, labeled untrusted content; only permitted data |
| Model | Safety tuning, system prompt |
| Output | PII/secret filters, grounding checks |
| Action | Allow-lists, parameter checks, approvals, budgets: decides whether manipulation achieves anything |
- Patterns: plan-then-execute, dual-LLM (quarantined reader without tools), fixed action sets, bounded loops.
- Fault injection: fail closed, reveal nothing, no half-finished actions.
Identity and authentication
- Every human, workload, agent, tool and service has an owner and suitable authentication.
- Agents are distinct non-human identities with a named human owner.
- Delegation, not impersonation: token names user (subject) and agent (actor), narrow scope (OAuth Token Exchange), bound, logged.
- M2M: workload identity federation first; static keys last (vaulted, rotated).
- Agents never see raw secrets: vault, runtime injection, credential broker.
Authorization
- Toxic combination: untrusted content + sensitive data + external communication in one agent → remove a leg.
- RBAC (coarse) · ABAC (data-dependent) · PBAC (per call) · JIT replaces standing privilege.
- Central PDP with PEPs at gateway, retrieval, tool broker, data stores. The model is never a PEP.
- Excessive agency: minimal tools, attenuation with expiry and depth caps, approvals, segmentation.
Identity lifecycle
- Inventory of AI identities: owner, purpose, credentials, permissions, activity. No owner / no activity / excess permission = finding.
- Deprovision every credential type: OAuth refresh tokens and separately issued keys survive account disablement.
- Per-identity token accounting = early detection.