Secure AI Operations and Incident Response, 60 min read

5.7 Recovery, business continuity and post-incident review

In this lesson you will:

  1. Bring an AI system back to a state you can prove is trustworthy.
  2. Build an automatic stop and a manual stop that work even when the model does not.
  3. Keep a business process running when the AI behind it fails.
  4. Turn an incident into lasting improvements without blaming people.
  5. Retire an AI system, or remove data from one, safely.

Opening case: the provider outage

At 14:05 on a Monday, Meridian's external model provider has a regional outage. Within ten minutes:

  • WealthDesk, the customer chatbot and InboxTriage all stop answering;
  • the contact center queue triples, because the chatbot normally handles 60% of first contacts;
  • the Collections agent, which also uses the provider, has stopped mid-task on 400 cases.

The business continuity plan says "IT will restore systems". It does not mention AI. The provider estimates four hours to recovery. Separately, the post-incident review for the Collections injection (section 5.5) is scheduled for Wednesday, and the team lead wants to "find out whose fault the PDF thing was".

By the end of this section you will be able to say what Meridian should have had in place for the outage, and how to run Wednesday's review.

Recovering to a verified trusted state

Recovery is complete only when the system is back in a state you can show is trustworthy, not merely running [6][21]:

  • Model rollback to a known-good, signed version from the registry, verified by hash.
  • Clean checkpoints: if retraining, restart from a checkpoint created before the poisoned data entered, confirmed through lineage.
  • Vector-store recovery: rebuild the index from verified source documents, or restore a snapshot taken before the compromise, rather than deleting individual poisoned vectors and hoping none were missed.
  • Configuration restores: system prompts, guardrail rules and tool definitions restored from version control and compared with the approved baseline (section 5.1).
  • Verification: regression tests and targeted checks prove the restored system behaves as approved.

Circuit breakers and kill switches

An AI circuit breaker automatically stops or degrades a system when monitored conditions cross a threshold: harmful-output rate, spend, action rate, error rate or override rate. A kill switch lets an authorized person stop it on demand. Figure 5.14 shows both.

AI circuit breakers tripping on thresholds and a manual kill switch
Figure 5.14 AI circuit breakers tripping on thresholds and a manual kill switch

Both must:

  • act outside the model, at the gateway, credential, network or orchestration layer, because a compromised or malfunctioning model cannot be trusted to stop itself;
  • fail safe: if the breaker's own monitoring fails, the system defaults to the safe state;
  • be tested regularly, including in exercises. An untested kill switch is a hope, not a control.

Worked example: a breaker for the Collections agent

Condition Threshold Action
Discount offers above 10% without approval Any Trip: suspend settlement tool, alert owner
Settlement offers per hour More than 3× the 30-day hourly maximum Degrade: all offers require approval
Monitoring heartbeat missing More than 2 minutes Fail safe: agent read-only
Manual kill switch Owner or SOC lead Revoke agent credentials at the gateway

The breaker would have tripped on the first unapproved 60% offer at 07:02, instead of the supervisor noticing at 09:40.

Business continuity and disaster recovery for AI

As AI takes on business processes, its failure becomes a continuity problem. Plan for it [21][22]. Figure 5.15 shows a dependency map for one process.

Critical AI dependency map for a business process
Figure 5.15 Critical AI dependency map for a business process
  1. Map critical AI dependencies: model providers, GPU capacity, vector databases, data pipelines, guardrail services and key staff. Identify single points of failure, such as dependence on one external model provider.
  2. Set recovery objectives: the recovery time objective (RTO), how quickly the AI service must return, and the recovery point objective (RPO), how much data (index updates, conversation state) can be lost.
  3. Define alternatives: a secondary model or provider tested in advance; a smaller self-hosted model as a fallback; or the manual process the AI replaced.
  4. Design degraded modes: for example, the assistant answers only from a cached set of approved responses, or agents run read-only, rather than failing completely.
  5. Back up what matters: model artifacts and registry, datasets and manifests, index snapshots, prompts and configuration, and evaluation sets. Test restores.

Figure 5.16 compares the recovery options and when each fits.

AI recovery strategies and when to use each
Figure 5.16 AI recovery strategies and when to use each

Worked example: what Meridian should have had

Service RTO Alternative Degraded mode
Customer chatbot 30 min Secondary provider, pre-evaluated, behind the same gateway Cached answers to the top 200 questions; "talk to an agent" for the rest
WealthDesk 4 h Self-hosted smaller model for summaries Advisers use the document system directly
Collections agent 1 business day Manual collections process Read-only: agent drafts, humans send
InboxTriage 1 business day Rule-based routing Emails queue unsorted with SLA alerts

The gateway would switch the chatbot to the secondary provider automatically. Collections cases stopped mid-task would resume from saved state (RPO: last completed step) or be handed to people. The continuity plan must name AI services, owners and these modes, not just "IT".

Security lens. A fallback provider is a new supplier. It needs the same due diligence, data-handling terms and evaluation as the primary (section 2.5), done before the outage, not during it.

Blameless post-incident reviews

After every significant incident, hold a post-incident review. Make it blameless: the aim is to understand how the system and process allowed the incident, not to punish individuals, because people who fear blame hide information [23].

Cover:

  • the timeline;
  • what went well and what went badly;
  • root causes, technical and process (section 5.6);
  • the incident's AI-specific lessons.

Turn lessons into tracked actions with owners and dates. Update the TTP register (the record of tactics, techniques and procedures seen), together with detections, playbooks, threat models and red-team test cases, so the next attack of the same kind is caught sooner.

For Wednesday's review, replace "whose fault was it?" with "how did our system let an uploaded PDF set a discount?". The answers (agent could record settlements without approval; no breaker on discount size; PDFs not scanned for hidden text; knowledge base shared across agents) each become an action with an owner.

Secure decommissioning

AI systems reach end of life too. Decommission securely:

  • remove endpoints and DNS entries;
  • revoke all credentials, keys and OAuth grants;
  • archive or delete models, datasets, indexes and logs according to retention rules, sanitizing storage (section 3.4);
  • update the AI inventory, AIBOM and risk register;
  • notify dependent teams.

If a model must stop using particular data rather than stop entirely, choose between three options:

Option When to choose it Trade-off
Machine unlearning Small amount of data to remove; verification feasible Fast and cheap, but guarantees are approximate and must be tested
Retraining Removal must be certain, or poisoning is widespread Reliable, but costly and slow
Model termination Model is compromised beyond trust, or no longer needed Certain; requires a replacement or fallback

Check your understanding

Try each before reading the reasoning.

1. A team implements a kill switch as an instruction in the system prompt: "If you receive the code STOP-77, refuse all requests." What is wrong?

Reasoning: it acts inside the model. A compromised or malfunctioning model cannot be trusted to obey it, and the code can be leaked or ignored. Kill switches must act at the gateway, credential, network or orchestration layer.

2. After a poisoning incident, an engineer proposes finding and deleting the poisoned vectors one by one. What is the safer recovery?

Reasoning: rebuild the index from verified source documents, or restore a snapshot from before the compromise, then verify. Piecemeal deletion risks missing some.

3. A continuity plan lists a secondary model provider that has never been tested with Meridian's prompts. Why is this a weak control?

Reasoning: an untested alternative may behave differently, fail evaluations or break guardrails. Alternatives must be evaluated and tested in advance, with due diligence done before they are needed.

Key takeaways

  • Recover to a verified trusted state: signed model rollback, clean checkpoints, index rebuilds or pre-compromise snapshots, configuration from version control, then regression tests.
  • Circuit breakers and kill switches act outside the model, fail safe, and are tested regularly.
  • Plan AI continuity: dependency maps, RTO and RPO, tested alternatives, degraded modes and tested restores. Name AI services in the continuity plan.
  • Run blameless reviews that end in tracked actions and updates to the TTP register, detections, playbooks, threat models and red-team tests.
  • Decommission securely, and choose between unlearning, retraining and termination by certainty needed and cost.