Secure AI Architecture, Engineering and IAM, 10 min read

Ready reckoner

Architecture principles

  • The model cannot enforce security; the architecture must. Zero Trust on every call, defense in depth, privacy by design, secure defaults.
  • Main control points: AI gateway and orchestrator/harness.
  • Threat-model inputs, retrieval, model, endpoints, tools/agents, memory, pipelines, outputs. Watch confused deputies.
  • Tools/MCP/A2A: least privilege, allow-lists, pinned definitions, validated calls and results, no token passthrough, human approval for high impact.

Infrastructure

  • ML tooling (notebooks, trackers) is the most common starting point: authenticate, never expose publicly.
  • Isolation ladder: process → container → hardened runtime → micro-VM → VM → dedicated → confidential computing. Agent-generated code = least trusted → sandbox, no network, no secrets, destroyed after each task.
  • Exhaustion controls: rate and token limits, quotas, timeouts, capped autoscaling. Clear GPU memory between jobs.

Pipelines (MLOps / LLMOps)

  • Prompts, indexes, guardrail configs and tool definitions are release artifacts. Whoever controls the pipeline controls production.
  • Ephemeral runners, federated short-lived credentials, pinned inputs, signed minimal images, protected pipeline definitions.
  • Data zones: raw → validated → curated (only the pipeline writes to curated).
  • Gates are enforceable via signed attestations + policy as code + separation of duties.

Guardrails

Layer Example
Input Injection/jailbreak classifiers, size limits
Context Separate, labeled untrusted content; only permitted data
Model Safety tuning, system prompt
Output PII/secret filters, grounding checks
Action Allow-lists, parameter checks, approvals, budgets: decides whether manipulation achieves anything
  • Patterns: plan-then-execute, dual-LLM (quarantined reader without tools), fixed action sets, bounded loops.
  • Fault injection: fail closed, reveal nothing, no half-finished actions.

Identity and authentication

  • Every human, workload, agent, tool and service has an owner and suitable authentication.
  • Agents are distinct non-human identities with a named human owner.
  • Delegation, not impersonation: token names user (subject) and agent (actor), narrow scope (OAuth Token Exchange), bound, logged.
  • M2M: workload identity federation first; static keys last (vaulted, rotated).
  • Agents never see raw secrets: vault, runtime injection, credential broker.

Authorization

  • Toxic combination: untrusted content + sensitive data + external communication in one agent → remove a leg.
  • RBAC (coarse) · ABAC (data-dependent) · PBAC (per call) · JIT replaces standing privilege.
  • Central PDP with PEPs at gateway, retrieval, tool broker, data stores. The model is never a PEP.
  • Excessive agency: minimal tools, attenuation with expiry and depth caps, approvals, segmentation.

Identity lifecycle

  • Inventory of AI identities: owner, purpose, credentials, permissions, activity. No owner / no activity / excess permission = finding.
  • Deprovision every credential type: OAuth refresh tokens and separately issued keys survive account disablement.
  • Per-identity token accounting = early detection.