5.7 Recovery, business continuity and post-incident review
In this lesson you will:
- Bring an AI system back to a state you can prove is trustworthy.
- Build an automatic stop and a manual stop that work even when the model does not.
- Keep a business process running when the AI behind it fails.
- Turn an incident into lasting improvements without blaming people.
- Retire an AI system, or remove data from one, safely.
Opening case: the provider outage
At 14:05 on a Monday, Meridian's external model provider has a regional outage. Within ten minutes:
- WealthDesk, the customer chatbot and InboxTriage all stop answering;
- the contact center queue triples, because the chatbot normally handles 60% of first contacts;
- the Collections agent, which also uses the provider, has stopped mid-task on 400 cases.
The business continuity plan says "IT will restore systems". It does not mention AI. The provider estimates four hours to recovery. Separately, the post-incident review for the Collections injection (section 5.5) is scheduled for Wednesday, and the team lead wants to "find out whose fault the PDF thing was".
By the end of this section you will be able to say what Meridian should have had in place for the outage, and how to run Wednesday's review.
Recovering to a verified trusted state
Recovery is complete only when the system is back in a state you can show is trustworthy, not merely running [6][21]:
- Model rollback to a known-good, signed version from the registry, verified by hash.
- Clean checkpoints: if retraining, restart from a checkpoint created before the poisoned data entered, confirmed through lineage.
- Vector-store recovery: rebuild the index from verified source documents, or restore a snapshot taken before the compromise, rather than deleting individual poisoned vectors and hoping none were missed.
- Configuration restores: system prompts, guardrail rules and tool definitions restored from version control and compared with the approved baseline (section 5.1).
- Verification: regression tests and targeted checks prove the restored system behaves as approved.
Circuit breakers and kill switches
An AI circuit breaker automatically stops or degrades a system when monitored conditions cross a threshold: harmful-output rate, spend, action rate, error rate or override rate. A kill switch lets an authorized person stop it on demand. Figure 5.14 shows both.
Both must:
- act outside the model, at the gateway, credential, network or orchestration layer, because a compromised or malfunctioning model cannot be trusted to stop itself;
- fail safe: if the breaker's own monitoring fails, the system defaults to the safe state;
- be tested regularly, including in exercises. An untested kill switch is a hope, not a control.
Worked example: a breaker for the Collections agent
| Condition | Threshold | Action |
|---|---|---|
| Discount offers above 10% without approval | Any | Trip: suspend settlement tool, alert owner |
| Settlement offers per hour | More than 3× the 30-day hourly maximum | Degrade: all offers require approval |
| Monitoring heartbeat missing | More than 2 minutes | Fail safe: agent read-only |
| Manual kill switch | Owner or SOC lead | Revoke agent credentials at the gateway |
The breaker would have tripped on the first unapproved 60% offer at 07:02, instead of the supervisor noticing at 09:40.
Business continuity and disaster recovery for AI
As AI takes on business processes, its failure becomes a continuity problem. Plan for it [21][22]. Figure 5.15 shows a dependency map for one process.
- Map critical AI dependencies: model providers, GPU capacity, vector databases, data pipelines, guardrail services and key staff. Identify single points of failure, such as dependence on one external model provider.
- Set recovery objectives: the recovery time objective (RTO), how quickly the AI service must return, and the recovery point objective (RPO), how much data (index updates, conversation state) can be lost.
- Define alternatives: a secondary model or provider tested in advance; a smaller self-hosted model as a fallback; or the manual process the AI replaced.
- Design degraded modes: for example, the assistant answers only from a cached set of approved responses, or agents run read-only, rather than failing completely.
- Back up what matters: model artifacts and registry, datasets and manifests, index snapshots, prompts and configuration, and evaluation sets. Test restores.
Figure 5.16 compares the recovery options and when each fits.
Worked example: what Meridian should have had
| Service | RTO | Alternative | Degraded mode |
|---|---|---|---|
| Customer chatbot | 30 min | Secondary provider, pre-evaluated, behind the same gateway | Cached answers to the top 200 questions; "talk to an agent" for the rest |
| WealthDesk | 4 h | Self-hosted smaller model for summaries | Advisers use the document system directly |
| Collections agent | 1 business day | Manual collections process | Read-only: agent drafts, humans send |
| InboxTriage | 1 business day | Rule-based routing | Emails queue unsorted with SLA alerts |
The gateway would switch the chatbot to the secondary provider automatically. Collections cases stopped mid-task would resume from saved state (RPO: last completed step) or be handed to people. The continuity plan must name AI services, owners and these modes, not just "IT".
Security lens. A fallback provider is a new supplier. It needs the same due diligence, data-handling terms and evaluation as the primary (section 2.5), done before the outage, not during it.
Blameless post-incident reviews
After every significant incident, hold a post-incident review. Make it blameless: the aim is to understand how the system and process allowed the incident, not to punish individuals, because people who fear blame hide information [23].
Cover:
- the timeline;
- what went well and what went badly;
- root causes, technical and process (section 5.6);
- the incident's AI-specific lessons.
Turn lessons into tracked actions with owners and dates. Update the TTP register (the record of tactics, techniques and procedures seen), together with detections, playbooks, threat models and red-team test cases, so the next attack of the same kind is caught sooner.
For Wednesday's review, replace "whose fault was it?" with "how did our system let an uploaded PDF set a discount?". The answers (agent could record settlements without approval; no breaker on discount size; PDFs not scanned for hidden text; knowledge base shared across agents) each become an action with an owner.
Secure decommissioning
AI systems reach end of life too. Decommission securely:
- remove endpoints and DNS entries;
- revoke all credentials, keys and OAuth grants;
- archive or delete models, datasets, indexes and logs according to retention rules, sanitizing storage (section 3.4);
- update the AI inventory, AIBOM and risk register;
- notify dependent teams.
If a model must stop using particular data rather than stop entirely, choose between three options:
| Option | When to choose it | Trade-off |
|---|---|---|
| Machine unlearning | Small amount of data to remove; verification feasible | Fast and cheap, but guarantees are approximate and must be tested |
| Retraining | Removal must be certain, or poisoning is widespread | Reliable, but costly and slow |
| Model termination | Model is compromised beyond trust, or no longer needed | Certain; requires a replacement or fallback |
Check your understanding
Try each before reading the reasoning.
1. A team implements a kill switch as an instruction in the system prompt: "If you receive the code STOP-77, refuse all requests." What is wrong?
Reasoning: it acts inside the model. A compromised or malfunctioning model cannot be trusted to obey it, and the code can be leaked or ignored. Kill switches must act at the gateway, credential, network or orchestration layer.
2. After a poisoning incident, an engineer proposes finding and deleting the poisoned vectors one by one. What is the safer recovery?
Reasoning: rebuild the index from verified source documents, or restore a snapshot from before the compromise, then verify. Piecemeal deletion risks missing some.
3. A continuity plan lists a secondary model provider that has never been tested with Meridian's prompts. Why is this a weak control?
Reasoning: an untested alternative may behave differently, fail evaluations or break guardrails. Alternatives must be evaluated and tested in advance, with due diligence done before they are needed.
Key takeaways
- Recover to a verified trusted state: signed model rollback, clean checkpoints, index rebuilds or pre-compromise snapshots, configuration from version control, then regression tests.
- Circuit breakers and kill switches act outside the model, fail safe, and are tested regularly.
- Plan AI continuity: dependency maps, RTO and RPO, tested alternatives, degraded modes and tested restores. Name AI services in the continuity plan.
- Run blameless reviews that end in tracked actions and updates to the TTP register, detections, playbooks, threat models and red-team tests.
- Decommission securely, and choose between unlearning, retraining and termination by certainty needed and cost.