AI vulnerability and posture management
In this lesson you will:
- Maintain model vulnerability and exposure registers
- Prioritise AI vulnerabilities by context and consequence
- Manage patching for AI pipelines, dependencies and open-source components
- Apply compensating controls and usage restrictions to legacy models
- Run continuous AI security reviews and posture-assurance metrics
Vulnerability management for AI covers more than CVEs. A model can be "vulnerable" to a jailbreak technique; a configuration can expose a vector database; a deprecated model can lack safety fixes. Posture management keeps all of it visible and prioritised.
1. Model vulnerability and exposure registers
A register records, for each AI system or model:
- Software vulnerabilities — CVEs in ML frameworks, serving runtimes, orchestration libraries, MCP servers, notebooks.
- Model weaknesses — known jailbreaks or injection techniques that work against it; red-team findings; bias findings; safety-evaluation failures.
- Exposures — internet-facing endpoints, public buckets, over-broad agent scopes, missing authentication.
- Supply-chain advisories — compromised models or datasets on public hubs; malicious packages.
- Status, owner, due date, compensating controls and residual-risk decisions.
Link entries to the AI inventory and AIBOMs so a new advisory immediately shows which systems are affected.
2. Context and consequence prioritisation
A raw severity score doesn't capture AI risk. Prioritise by:
| Factor | Question |
|---|---|
| Exposure | Is the system internet-facing? Who can reach it? |
| Data | What data can it read? Personal, financial, regulated? |
| Agency | Can it take actions? Irreversible ones? |
| Exploitability | Public exploit or jailbreak? Automatable? |
| Consequence | Financial loss, harm to people, regulatory breach, IP loss? |
| Compensating controls | Do guardrails or architecture already block the impact? |
A "medium" jailbreak against an internal drafting tool with no tools may be low priority; the same jailbreak against a public agent with payment tools is urgent.
3. Patch management for AI
- Dependencies — ML frameworks, tokenisers, serving runtimes and orchestration libraries move fast and have had serious vulnerabilities (including code execution on model load). Track them via SBOM/AIBOM and patch through the pipeline.
- Models — "patching" a model may mean updating to a provider's new version, retraining, or adding guardrails. All go through evaluation gates and staged rollout.
- Prompts and guardrails — updated to counter new jailbreak techniques; versioned and regression-tested.
- Open-source components — monitor maintainers and advisories; pin versions; mirror approved packages.
- Platform — GPU drivers, container images, notebook servers.
4. Legacy models: compensating controls and usage restrictions
Some models can't be retrained or replaced quickly — the original team has left, the data is gone, or a vendor has stopped supporting it.
- Wrap it — put a gateway in front with input validation, output filtering, rate limits and logging.
- Restrict its use — limit to internal users, specific decisions, lower-risk tiers; remove tool access.
- Add human review for decisions it influences.
- Monitor more closely — tighter drift and anomaly thresholds.
- Record the exception — risk acceptance with owner and expiry, and a replacement plan.
5. Continuous review and posture assurance
AI security posture management (AI-SPM) tools continuously discover AI assets (models, endpoints, agents, vector stores, AI services) and check them against policy: exposure, authentication, encryption, scopes, approved models, logging.
Posture-assurance metrics:
| Metric | What it shows |
|---|---|
| % of AI systems in inventory with owner and risk tier | Governance coverage |
| % of production models loaded from the approved registry with valid signatures | Integrity coverage |
| Mean time to remediate critical AI vulnerabilities | Response capability |
| % of agents with least-privilege scopes reviewed in last 90 days | Agency control |
| % of high-risk systems red-teamed in last 12 months | Assurance coverage |
| Unregistered AI services discovered per month | Shadow-AI trend |
| Guardrail regression pass rate | Defensive health |
Worked example — prioritising one advisory across three systems
A new jailbreak technique is published that bypasses safety training in a widely used model family.
| System | Exposure | Agency | Data | Priority |
|---|---|---|---|---|
| Internal meeting summariser | Staff only | None | Meeting notes | Low — monitor, update prompt defences in next cycle |
| Public marketing chatbot | Internet | None | Public content | Medium — reputational harm; add jailbreak pattern to input filter this week |
| Customer-service agent with refund tool | Internet | Refunds ≤ ₹2,000 | Customer orders | High — test within 24 hours; approval gates already limit impact; tighten thresholds pending provider fix |
🔐 Security lens
Track jailbreak techniques like CVEs: record them, test your systems against them, and retest when models or prompts change.
Key takeaways
- Keep one register for software CVEs, model weaknesses, exposures and supply-chain advisories — linked to the inventory and AIBOMs.
- Prioritise by exposure, data, agency, exploitability, consequence and existing controls — not raw scores.
- Patch dependencies, models, prompts, guardrails, open-source components and platforms through gated pipelines.
- Wrap and restrict legacy models; record exceptions with expiry and replacement plans.
- Use AI-SPM and posture metrics to show coverage and trends.
Self-check
- Why might the same jailbreak be low priority for one system and high for another?
- Name three compensating controls for a legacy model.
- Which metric would show progress against shadow AI?