Model governance, model cards and approved registries
In this lesson you will:
- Define model ownership, risk classification and approval workflows
- Write and use model cards as governance artefacts
- Operate an approved model registry as a security control
- Monitor models across their security lifecycle
Module 2 governed AI use cases. This module zooms in on the model itself — an asset that is valuable, fragile and easy to copy. Governance starts with knowing which models you have, who owns each, and which are allowed into production.
1. Model governance essentials
| Element | What it means |
|---|---|
| Ownership | Every model has a named owner accountable for its performance, risk and retirement — plus a technical owner who maintains it |
| Risk classification | Each model inherits the highest risk tier of the use cases it serves; a general model used in a Tier 3 decision is a Tier 3 model |
| Approval workflow | Defined gates — development → validation → approval → production — with evidence required at each |
| Independent validation | For higher tiers, someone other than the developer validates performance, robustness and fairness (the model-risk-management discipline used in banking) |
| Inventory linkage | Model records link to use-case records, datasets (lineage) and AIBOMs |
2. Model cards
A model card is a short, structured document describing a model: what it is for, how it was built and evaluated, and where it should not be used. Originally proposed as a transparency tool, it is now a core governance artefact — and the EU AI Act's technical-documentation duties cover much of the same ground.
Typical sections:
- Model details — name, version, owner, type, date, licence.
- Intended use — primary uses and users; out-of-scope uses.
- Training data — sources, dates, preprocessing, known gaps (link to datasheets).
- Evaluation — metrics, test sets, results overall and by subgroup.
- Robustness and security — adversarial testing performed, known weaknesses.
- Ethical considerations and limitations — bias findings, failure modes, caveats.
- Operational information — dependencies, hardware, monitoring thresholds, retraining cadence.
For generative systems, providers also publish system cards covering the whole system — model plus safeguards, tools and policies.
3. Approved model registries
A model registry stores model artefacts with their versions, metadata and status. An approved registry adds a control: production systems may only load models whose status is approved.
Registry security features:
- Status and stage — development, staging, approved, deprecated, revoked.
- Immutable versions — an approved version cannot be overwritten; changes create new versions.
- Integrity metadata — hash and signature for every artefact (Lesson 5.3).
- Access control — who can register, promote and download; promotion to approved requires a different role from the developer.
- Lineage links — training data versions, code commit, training run, evaluation results.
- Revocation — a revoked model is blocked at load time across all environments.
Third-party and open models enter the registry only after the provenance checks in Module 3 — making the registry the single gate for all models, built or bought.
4. Security lifecycle monitoring
After approval, models need watching:
- Performance and drift — accuracy against fresh labels, input distribution shifts, output distribution shifts.
- Bias tracking — fairness metrics by subgroup over time.
- Anomaly detection — unusual input patterns (possible evasion or extraction attempts), unusual query volumes, spikes in low-confidence outputs.
- Security signals — guardrail triggers, injection attempts, abuse reports.
Thresholds breached → alert the owner → investigate → retrain, roll back or restrict.
Worked example — a registry stops a risky shortcut
A developer under deadline pressure wants to deploy a newly fine-tuned credit model straight from a notebook.
| Control | What happened |
|---|---|
| Production loader | Refuses to load any model not marked approved in the registry |
| Registry promotion | Developer can register the model, but promotion needs the model-validation role |
| Validation | Independent validator finds approval rates for one region dropped 12 points versus the current model |
| Model card | Updated with the finding; out-of-scope note added |
| Outcome | Model held in staging; data issue traced to a missing region in the fine-tuning set; fixed and re-validated a week later |
🔐 Security lens
Enforce the registry at load time in production (allow-list by hash and status), not just by policy. Otherwise a model file copied onto a server bypasses every governance gate.
Key takeaways
- Every model needs an owner, a risk classification inherited from its use cases, and an approval workflow with independent validation for higher tiers.
- Model cards document intended and out-of-scope use, data, evaluations, robustness and limitations.
- An approved registry with immutable versions, integrity metadata, separation of duties and revocation is the single gate for built and bought models.
- Monitor drift, bias, anomalies and security signals after approval.
Self-check
- A general-purpose model serves both a Tier 1 and a Tier 3 use case. What tier is the model?
- Which model-card section should state that a model must not be used for employment decisions?
- Why should the developer not be able to promote their own model to approved?