AI Security Operations, Incident Response & Recovery, 25 min read

AI vulnerability and posture management

In this lesson you will:

  • Maintain model vulnerability and exposure registers
  • Prioritise AI vulnerabilities by context and consequence
  • Manage patching for AI pipelines, dependencies and open-source components
  • Apply compensating controls and usage restrictions to legacy models
  • Run continuous AI security reviews and posture-assurance metrics

Vulnerability management for AI covers more than CVEs. A model can be "vulnerable" to a jailbreak technique; a configuration can expose a vector database; a deprecated model can lack safety fixes. Posture management keeps all of it visible and prioritised.

1. Model vulnerability and exposure registers

AI vulnerability and exposure register entries.
Figure 8.7 An AI vulnerability and exposure register combines software CVEs, model weaknesses and configuration exposures in one view.

A register records, for each AI system or model:

  • Software vulnerabilities — CVEs in ML frameworks, serving runtimes, orchestration libraries, MCP servers, notebooks.
  • Model weaknesses — known jailbreaks or injection techniques that work against it; red-team findings; bias findings; safety-evaluation failures.
  • Exposures — internet-facing endpoints, public buckets, over-broad agent scopes, missing authentication.
  • Supply-chain advisories — compromised models or datasets on public hubs; malicious packages.
  • Status, owner, due date, compensating controls and residual-risk decisions.

Link entries to the AI inventory and AIBOMs so a new advisory immediately shows which systems are affected.

2. Context and consequence prioritisation

A raw severity score doesn't capture AI risk. Prioritise by:

Factor Question
Exposure Is the system internet-facing? Who can reach it?
Data What data can it read? Personal, financial, regulated?
Agency Can it take actions? Irreversible ones?
Exploitability Public exploit or jailbreak? Automatable?
Consequence Financial loss, harm to people, regulatory breach, IP loss?
Compensating controls Do guardrails or architecture already block the impact?

A "medium" jailbreak against an internal drafting tool with no tools may be low priority; the same jailbreak against a public agent with payment tools is urgent.

3. Patch management for AI

  • Dependencies — ML frameworks, tokenisers, serving runtimes and orchestration libraries move fast and have had serious vulnerabilities (including code execution on model load). Track them via SBOM/AIBOM and patch through the pipeline.
  • Models — "patching" a model may mean updating to a provider's new version, retraining, or adding guardrails. All go through evaluation gates and staged rollout.
  • Prompts and guardrails — updated to counter new jailbreak techniques; versioned and regression-tested.
  • Open-source components — monitor maintainers and advisories; pin versions; mirror approved packages.
  • Platform — GPU drivers, container images, notebook servers.

4. Legacy models: compensating controls and usage restrictions

Some models can't be retrained or replaced quickly — the original team has left, the data is gone, or a vendor has stopped supporting it.

Compensating controls wrapped around a legacy model.
Figure 8.8 Wrapping a legacy model in compensating controls and restricting its use until it can be replaced.
  • Wrap it — put a gateway in front with input validation, output filtering, rate limits and logging.
  • Restrict its use — limit to internal users, specific decisions, lower-risk tiers; remove tool access.
  • Add human review for decisions it influences.
  • Monitor more closely — tighter drift and anomaly thresholds.
  • Record the exception — risk acceptance with owner and expiry, and a replacement plan.

5. Continuous review and posture assurance

AI security posture management (AI-SPM) tools continuously discover AI assets (models, endpoints, agents, vector stores, AI services) and check them against policy: exposure, authentication, encryption, scopes, approved models, logging.

Posture-assurance metrics:

Metric What it shows
% of AI systems in inventory with owner and risk tier Governance coverage
% of production models loaded from the approved registry with valid signatures Integrity coverage
Mean time to remediate critical AI vulnerabilities Response capability
% of agents with least-privilege scopes reviewed in last 90 days Agency control
% of high-risk systems red-teamed in last 12 months Assurance coverage
Unregistered AI services discovered per month Shadow-AI trend
Guardrail regression pass rate Defensive health

Worked example — prioritising one advisory across three systems

A new jailbreak technique is published that bypasses safety training in a widely used model family.

System Exposure Agency Data Priority
Internal meeting summariser Staff only None Meeting notes Low — monitor, update prompt defences in next cycle
Public marketing chatbot Internet None Public content Medium — reputational harm; add jailbreak pattern to input filter this week
Customer-service agent with refund tool Internet Refunds ≤ ₹2,000 Customer orders High — test within 24 hours; approval gates already limit impact; tighten thresholds pending provider fix

🔐 Security lens

Track jailbreak techniques like CVEs: record them, test your systems against them, and retest when models or prompts change.

Key takeaways

  • Keep one register for software CVEs, model weaknesses, exposures and supply-chain advisories — linked to the inventory and AIBOMs.
  • Prioritise by exposure, data, agency, exploitability, consequence and existing controls — not raw scores.
  • Patch dependencies, models, prompts, guardrails, open-source components and platforms through gated pipelines.
  • Wrap and restrict legacy models; record exceptions with expiry and replacement plans.
  • Use AI-SPM and posture metrics to show coverage and trends.

Self-check

  1. Why might the same jailbreak be low priority for one system and high for another?
  2. Name three compensating controls for a legacy model.
  3. Which metric would show progress against shadow AI?