AI SecOps foundations and operational readiness
In this lesson you will:
- Apply least privilege and separation of duties to AI operations
- Define AI operational roles and responsibilities
- Set release requirements such as human-override baselines and reject-rate thresholds
- Prepare playbooks, runbooks and tabletop exercises for AI incidents
- Manage configuration, automated model updates and configuration drift
Security operations for AI starts before the first alert. Teams need clear roles, release criteria that define "normal", rehearsed playbooks and control over configuration — because in AI, configuration is behaviour.
1. Core principles in AI operations
- Least privilege — operators get only the access needed: viewing telemetry doesn't require changing prompts; changing guardrails doesn't require downloading weights.
- Separation of duties — the person who changes a model, prompt or guardrail is not the person who approves it; incident responders who can roll back models are distinct from developers who can deploy new ones, with break-glass procedures for emergencies.
- Need to know for telemetry — prompt and output logs contain sensitive content; access is restricted and logged.
2. Roles and responsibilities
| Role | Responsibility |
|---|---|
| Model / system owner | Accountable for the AI system in production; decides on rollback, suspension and risk acceptance |
| Data owner | Accountable for data used in training and retrieval; decides on data quarantine and purges |
| MLOps / platform engineers | Operate pipelines, registries, serving infrastructure; execute rollbacks and redeployments |
| SOC analysts | Monitor AI telemetry, triage alerts, run playbooks |
| AI security engineers | Detection engineering for AI, threat modelling, red-team liaison |
| Data scientists | Diagnose model behaviour, retrain, evaluate fixes |
| Privacy, legal, compliance | Breach assessment, regulatory notification, evidence handling |
| Communications | Customer and public statements |
3. Release requirements as operational baselines
Release criteria (Module 5) double as the baseline that operations monitors against:
- Quality thresholds — accuracy, groundedness and task success rates.
- Safety thresholds — jailbreak-suite pass rate; harmful-output rate.
- Reject-rate thresholds — the expected rate at which guardrails block inputs or outputs. A sudden rise can mean an attack (or a bad change); a sudden fall can mean a guardrail stopped working.
- Human-override baselines — the expected rate at which reviewers override AI decisions in HITL processes. Large deviations in either direction need investigation.
- Latency, cost and volume baselines — per identity and in aggregate.
4. Playbooks, runbooks and tabletop exercises
- Playbooks describe the response to an incident type: triggers, roles, decisions, escalation, communications.
- Runbooks are the technical steps: how to roll back a model, disable a tool, rotate a provider key, rebuild a vector index, suspend an agent.
AI incident types that deserve their own playbooks:
| Playbook | Key decisions |
|---|---|
| Prompt injection / data exfiltration via assistant | Disable affected tools; restrict retrieval; notify privacy |
| Data poisoning | Quarantine data source; identify affected models via lineage; roll back |
| Model tampering or theft | Verify artefact integrity; revoke access; assess IP and legal impact |
| Agent misuse / unauthorised actions | Suspend agent; revoke tokens; reverse actions |
| Harmful or biased outputs at scale | Restrict or suspend; communications; regulatory assessment |
| Provider incident or model change | Switch to fallback; pin versions |
| Denial of wallet / resource abuse | Throttle; block identities; enforce caps |
Tabletop exercises rehearse these with the real people. Good AI scenarios: "Monday morning, the fraud model's approval rate for one region has doubled since Friday's retrain" (poisoning or bad data?); "A customer posts screenshots of your assistant revealing another customer's address" (injection or ACL failure?).
5. Configuration management and drift
AI behaviour is defined by configuration as much as by code: model versions, prompts, temperatures, tool lists, scopes, guardrail thresholds, retrieval sources.
- Store all of it as code in version control; deploy through pipelines; review changes.
- Automated model updates — whether a provider's silent update or your own scheduled retrain — go through the same gates and staged rollout. Pin versions where possible.
- Configuration drift tracking — compare running configuration against the approved baseline continuously; alert on differences (a guardrail threshold changed in the console, a tool added to an agent outside the pipeline).
Worked example — a tabletop exercise on model tampering
Scenario: the SOC sees the production credit model's hash differs from the registry's approved hash after a weekend maintenance window.
| Inject | Expected decision |
|---|---|
| Hash mismatch alert | Who owns this? (system owner) — invoke model-tampering playbook |
| Approval rates unchanged so far | Contain anyway: roll back to the signed approved version; preserve the suspect artefact |
| Maintenance account used outside its change ticket | Disable account; preserve logs; involve insider-risk team |
| Decisions made with the suspect model: 3,400 | Data scientists compare decisions; legal assesses customer impact |
Gaps found: no runbook for preserving a model artefact forensically; break-glass rollback required a developer who was on leave. Both fixed within a month.
🔐 Security lens
A reject rate that drops to zero is as alarming as one that spikes — it can mean an attacker found a bypass, or a misconfiguration disabled the guardrail. Alert on both directions.
Key takeaways
- Apply least privilege, separation of duties and need-to-know to AI operations and telemetry.
- Define owners for systems and data, and responsibilities for MLOps, SOC, AI security, data science, privacy and communications.
- Release thresholds become operational baselines: quality, safety, reject rates, override rates, latency, cost.
- Prepare AI-specific playbooks and runbooks; rehearse them in tabletop exercises.
- Manage AI configuration as code; gate automated updates; detect configuration drift.
Self-check
- Why is a sudden fall in guardrail reject rate a signal worth investigating?
- Give two AI incident types that need their own playbooks.
- What is configuration drift in an AI system?