The secure AI development lifecycle and change management
In this lesson you will:
- Integrate security across the AI development lifecycle (AIDLC)
- Adapt environment separation, least privilege and secure API integration for AI
- Manage change across models, datasets, prompts and retrieval sources
- Use staged rollouts, canary deployments and rollback procedures
Secure software development lifecycles already exist. The secure AI development lifecycle (AIDLC) keeps them and adds activities for data, models and behaviour — and treats more things as "changes" than traditional change management does.
1. Security across the AIDLC
| Phase | Security activities |
|---|---|
| Plan | Use-case intake and tiering; impact assessment; security and privacy requirements; abuse cases |
| Data | Provenance, classification, validation, privacy techniques, lineage |
| Design | Threat model (Module 7), architecture review, tool/permission design, guardrails |
| Build and train | Secure coding, dependency control, secure pipelines, fine-tuning lineage |
| Evaluate | Performance, fairness, robustness, safety and red-team testing |
| Deploy | Signed artefacts, admission control, staged rollout, monitoring set-up |
| Operate | Telemetry, detection, incident response, drift and bias monitoring |
| Retire | Decommissioning, data and model disposal, access revocation |
2. Environment controls adapted for AI
- Separation — development, test and production environments separated, with production data kept out of development. If realistic data is needed, use masked or synthetic data.
- Least privilege — data scientists get access to the data a project needs, for as long as needed; notebooks do not hold production credentials; agents in test cannot reach production tools.
- Secure API integration — model and tool APIs called through gateways with authentication, quotas and logging; no direct calls to external model APIs from user devices with shared keys.
- Egress control — training and serving environments restricted from arbitrary internet access, reducing exfiltration and malicious-package risk.
3. What counts as a change in AI
Traditional change management focuses on code and infrastructure. AI behaviour can change through many other routes:
| Change | Example | Typical control |
|---|---|---|
| Model version | New base model, provider update, fine-tune | Full evaluation suite, staged rollout, approval by tier |
| Dataset version | New training or evaluation data | Data gates, lineage, retrain evaluation |
| Prompt / system instructions | Edited system prompt or templates | Versioning, regression tests on safety and quality |
| Retrieval sources | New document collection indexed | Source approval, classification check, access-filter test |
| Tools and permissions | Agent gets a new tool or wider scope | Security review, threat model update |
| Guardrail configuration | Filter thresholds changed | Regression testing; second-person review |
| Parameters | Temperature, max tokens | Lightweight review and testing |
Versioning all of these together — a "release" that pins model, prompt, tools, guardrails and index versions — is what makes rollback possible.
4. Staged rollouts and rollback
- Shadow deployment — the new model receives copies of real traffic, but its outputs are not used; compare against production.
- Canary deployment — a small percentage (for example 1–5%) of users get the new version; widen gradually.
- A/B testing — controlled comparison of versions on business and quality metrics.
- Feature flags — switch behaviour on or off without redeploying.
- Automated rollback — if canary guardrail metrics (error rate, safety filter hits, complaint rate, latency) breach thresholds, traffic returns to the previous version automatically.
Worked example — a prompt change that needed a rollback
A team edits the system prompt of a banking assistant to make answers "friendlier". It is released as a minor change.
| Day | Observation |
|---|---|
| Day 1 (5% canary) | Satisfaction up slightly; safety-filter hits up 40% |
| Analysis | New tone instructions weakened the refusal wording; jailbreak test suite pass rate dropped from 98% to 91% |
| Action | Automatic rollback triggered by the safety-hit threshold; prompt classed as a behavioural change requiring the safety gate |
| Fix | Revised prompt keeps refusal instructions; passes the suite at 98%; rolled out 5% → 25% → 100% over a week |
🔐 Security lens
Ask "what changed?" during every AI incident — and make sure the answer covers prompts, tools, guardrails, indexes and provider model versions, not just code deploys.
Key takeaways
- The secure AIDLC adds data, model, evaluation and behavioural activities to a classic SDLC, from planning to retirement.
- Separate environments, keep production data out of development, apply least privilege and route model APIs through gateways.
- Model, dataset, prompt, retrieval, tool, guardrail and parameter changes all alter behaviour — version them together as a release.
- Use shadow, canary and A/B rollouts with automated rollback on guardrail metrics.
Self-check
- Give three AI changes that would not appear in a traditional code-only change log.
- What is the difference between a shadow deployment and a canary deployment?
- Why should prompt edits pass a safety regression gate?