Every week, business teams propose new uses of AI: a chatbot for customers, a model that scores leads, a coding assistant, an agent that processes invoices. Security and risk teams are asked to "approve" them, often with little time and no agreed method. Here is a five-step assessment that works for most use cases and lines up with the NIST AI Risk Management Framework and ISO/IEC 42001.
Step 1: understand the context
Before looking at the technology, answer:
- Purpose. What decision or task does the AI support? What happens today without it?
- Users and affected people. Who uses it, and who is affected by its outputs? Customers, employees, the public, children?
- Autonomy. Does it advise a human, or act on its own?
- Stakes. What happens if it's wrong? A bad product description is very different from a wrong credit decision.
- Regulation. Do privacy law, sector rules or the EU AI Act apply? Is it a high-risk use?
This step alone sorts most use cases into low, medium and high scrutiny.
Step 2: map the system and its data
Draw the components:
- Model: built in-house, fine-tuned, or a third-party API or open-weight model? Which version?
- Data: training and fine-tuning data, retrieval sources, user inputs and outputs. Does any of it include personal or confidential data? Where is it processed and stored? Does the provider use it for training?
- Integrations: tools, APIs and systems the AI can read from or act on.
- People: who builds it, operates it, reviews outputs and handles complaints.
Step 3: identify the risks
Work through the main categories:
| Category | Questions to ask |
|---|---|
| Security | Prompt injection, data leakage, model or data poisoning, excessive agency, insecure output handling, supply chain (see prompt injection explained) |
| Privacy | Personal data in prompts and training, retention, purpose limitation, individuals' rights, cross-border transfer |
| Accuracy and reliability | Hallucination, performance on edge cases, drift over time |
| Fairness | Different error rates for different groups; biased training data |
| Transparency | Do users know they are dealing with AI? Can decisions be explained? |
| Accountability | Who owns the outcome? Is there human oversight and an appeal route? |
| Legal and IP | Copyright in training data and outputs; contractual restrictions; regulatory obligations |
| Operational | Vendor dependency, cost overruns, availability |
Rate each relevant risk on likelihood and impact, as you would any other risk. Weight impact towards harm to people, not just harm to the company.
Step 4: choose controls
Match controls to the significant risks. Common ones:
- Data controls: minimise personal data in prompts, mask sensitive fields, choose providers that don't train on your data, set retention.
- Access and privilege controls: least-privilege tool access, the user's own permissions, and confirmation steps for actions.
- Quality controls: evaluation sets before launch, human review for high-stakes outputs, and confidence thresholds.
- Fairness testing on relevant groups where decisions affect people.
- Transparency: AI disclosure, explanations and a route to a human.
- Security testing: red-teaming, including indirect injection.
- Contractual controls with AI vendors: data use, security, incident notice and model change notice.
Record the residual risk and get sign-off from the business owner at the right level.
Step 5: monitor and review
AI systems change: models are updated, data drifts, users find new uses. Set up:
- monitoring of quality metrics, complaints, unusual usage and security events;
- triggers for reassessment: a new model version, new data source, new user group or new capability such as tool access;
- incident handling for AI failures, linked to your security and privacy incident processes;
- a register of approved AI use cases with owners and review dates.
Fit it into governance
The assessment works best inside a lightweight AI governance process: an intake form, a triage by risk level, assessment templates, an approval forum for high-risk cases, and an inventory. ISO/IEC 42001 provides a management-system structure for this. The NIST AI RMF's four functions (Govern, Map, Measure, Manage) map neatly onto the steps above.
Learn more
Our free AI Security & Governance course walks through AI risk assessment, governance frameworks and controls, with a final exam and certificate. For privacy-heavy use cases, combine this with a DPIA.
Written and reviewed by the DigiTrust Asia editorial team. How we research, review and correct articles.
General information, not legal advice. Laws and official guidance change; check the current text before acting.
Build AI governance skills
AI Security & Governance is a free course with a final exam and certificate.
Create a free account