Every week, business teams propose new uses of AI: a chatbot for customers, a model that scores leads, a coding assistant, an agent that processes invoices. Security and risk teams are asked to "approve" them, often with little time and no agreed method. Here is a five-step assessment that works for most use cases and lines up with the NIST AI Risk Management Framework and ISO/IEC 42001.

Step 1: understand the context

Before looking at the technology, answer:

  • Purpose. What decision or task does the AI support? What happens today without it?
  • Users and affected people. Who uses it, and who is affected by its outputs? Customers, employees, the public, children?
  • Autonomy. Does it advise a human, or act on its own?
  • Stakes. What happens if it's wrong? A bad product description is very different from a wrong credit decision.
  • Regulation. Do privacy law, sector rules or the EU AI Act apply? Is it a high-risk use?

This step alone sorts most use cases into low, medium and high scrutiny.

Step 2: map the system and its data

Draw the components:

  • Model: built in-house, fine-tuned, or a third-party API or open-weight model? Which version?
  • Data: training and fine-tuning data, retrieval sources, user inputs and outputs. Does any of it include personal or confidential data? Where is it processed and stored? Does the provider use it for training?
  • Integrations: tools, APIs and systems the AI can read from or act on.
  • People: who builds it, operates it, reviews outputs and handles complaints.

Step 3: identify the risks

Work through the main categories:

Category Questions to ask
Security Prompt injection, data leakage, model or data poisoning, excessive agency, insecure output handling, supply chain (see prompt injection explained)
Privacy Personal data in prompts and training, retention, purpose limitation, individuals' rights, cross-border transfer
Accuracy and reliability Hallucination, performance on edge cases, drift over time
Fairness Different error rates for different groups; biased training data
Transparency Do users know they are dealing with AI? Can decisions be explained?
Accountability Who owns the outcome? Is there human oversight and an appeal route?
Legal and IP Copyright in training data and outputs; contractual restrictions; regulatory obligations
Operational Vendor dependency, cost overruns, availability

Rate each relevant risk on likelihood and impact, as you would any other risk. Weight impact towards harm to people, not just harm to the company.

Step 4: choose controls

Match controls to the significant risks. Common ones:

  • Data controls: minimise personal data in prompts, mask sensitive fields, choose providers that don't train on your data, set retention.
  • Access and privilege controls: least-privilege tool access, the user's own permissions, and confirmation steps for actions.
  • Quality controls: evaluation sets before launch, human review for high-stakes outputs, and confidence thresholds.
  • Fairness testing on relevant groups where decisions affect people.
  • Transparency: AI disclosure, explanations and a route to a human.
  • Security testing: red-teaming, including indirect injection.
  • Contractual controls with AI vendors: data use, security, incident notice and model change notice.

Record the residual risk and get sign-off from the business owner at the right level.

Step 5: monitor and review

AI systems change: models are updated, data drifts, users find new uses. Set up:

  • monitoring of quality metrics, complaints, unusual usage and security events;
  • triggers for reassessment: a new model version, new data source, new user group or new capability such as tool access;
  • incident handling for AI failures, linked to your security and privacy incident processes;
  • a register of approved AI use cases with owners and review dates.

Fit it into governance

The assessment works best inside a lightweight AI governance process: an intake form, a triage by risk level, assessment templates, an approval forum for high-risk cases, and an inventory. ISO/IEC 42001 provides a management-system structure for this. The NIST AI RMF's four functions (Govern, Map, Measure, Manage) map neatly onto the steps above.

Learn more

Our free AI Security & Governance course walks through AI risk assessment, governance frameworks and controls, with a final exam and certificate. For privacy-heavy use cases, combine this with a DPIA.

Written and reviewed by the DigiTrust Asia editorial team. How we research, review and correct articles.

General information, not legal advice. Laws and official guidance change; check the current text before acting.

Build AI governance skills

AI Security & Governance is a free course with a final exam and certificate.

Create a free account