Ready reckoner
Model governance
- Every model: owner, risk tier = highest tier of its use cases, approval workflow, independent validation for higher tiers.
- Model card: details · intended use · out-of-scope uses · training data · evaluation (by subgroup) · robustness & security · limitations · operations. System card = whole GenAI system.
- Approved registry: status/stage · immutable versions · hash + signature · separation of duties for promotion · lineage · revocation. Enforce at load time.
- Monitor: drift · bias · anomalies (systematic querying) · security signals.
Confidentiality
| Protect | How |
|---|---|
| Weights at rest | Encryption, own keys, least privilege, download alerts |
| In transit | TLS, signed artefacts |
| Serving | Isolated hosts, no interactive access |
| Edge | Encrypt/obfuscate; assume extractable |
| Partners | TEE + attestation, contract |
- Extraction = query API → train surrogate. Controls: rate limits, quotas, pattern monitoring, labels not probabilities, verified accounts, terms.
- Surrogates enable offline adversarial crafting (transferability).
- Watermarking (prove ownership) · fingerprinting (detect derivation, no modification) · output watermarking (mark generated content). All probabilistic.
Integrity and availability
- Hash + sign at approval, verify at load. Version weights, tokenizer, adapters, prompts, tool defs, guardrails, index.
- Safe formats; read-only volumes; immutable images.
- Output is untrusted input downstream → escape, parameterise, least-privilege roles (OWASP improper output handling).
- Keep previous version deployable; rehearse rollback; pin hosted versions.
- Availability: rate limits · quotas · input/output caps · timeouts · capacity isolation · graceful degradation · multi-provider fallback.
- Audit trails: events + references; content masked and restricted.
Secure MLOps / LLMOps
Stages: source → ingest → train → evaluate → package → registry → deploy (admission control). Gates: data · quality · fairness · safety/security · supply chain · approval. Bypass only via documented exception.
- Minimal signed non-root images; SBOM/AIBOM at build.
- Fine-tune lineage: base · datasets · code · who/where · outputs · safety results.
- Credentials: workload identity, short-lived, scoped; build ≠ deploy identity.
- Protect evaluation sets from training writes. Reproducible training enables unlearning.
Change management
AIDLC: plan · data · design · build · evaluate · deploy · operate · retire. AI changes: model · dataset · prompt · retrieval sources · tools & permissions · guardrails · parameters → version together as one release. Rollouts: shadow (no user impact) · canary (small %) · A/B · flags · auto-rollback on guardrail metrics. Environments: separate; no production personal data in dev; least privilege; gateways for model APIs; egress control.
Exam-day reflexes
- "Model serves Tier 1 and Tier 3" → Tier 3.
- "Copied model file bypassed governance" → load-time allow-listing.
- "Steal without weights" → extraction; return labels only.
- "SQL from the model run as admin" → least-privilege role + allow-list.
- "Prompt tone edit" → behavioural change, safety regression gate.