The first wave of generative AI produced chatbots: you ask, they answer. The current wave produces agents: systems that plan multi-step tasks and carry them out by calling tools, such as searching, reading files, sending messages, querying databases, running code or calling other agents. Agents are far more useful, and far riskier, because they act.

Why agents change the threat model

With a chatbot, the worst case is usually a harmful or wrong answer. With an agent, the worst case is a harmful action: data sent to the wrong place, records deleted, money moved, code deployed. And agents take in content from many sources, any of which may contain prompt injection.

OWASP calls this excessive agency: an LLM-based system with more functionality, permissions or autonomy than it needs.

The confused-deputy problem

A confused deputy is a program with legitimate authority that is tricked into using it on someone else's behalf. Agents are natural confused deputies:

  • an agent with access to the whole company's documents is asked by a junior employee to "summarise the board papers";
  • an email agent reads a message from an outsider that instructs it to forward internal files;
  • an agent calling another agent passes along instructions that the second agent trusts because they come from "inside".

The fix is about whose authority the agent uses.

Controls that matter

1. Give agents an identity

Treat each agent as a non-human identity with its own credentials. Don't let it share a human's password or a generic service account. Register agents in your identity platform, give them an owner, and include them in access reviews.

2. Act on behalf of the user, with the user's rights

When an agent acts for a user, it should use delegated, scoped authorisation, for example OAuth tokens issued for that user and that task, so it can never exceed the user's permissions. An agent should not be a way around access control.

3. Least privilege for tools

  • Expose only the tools the task needs. A research agent doesn't need email-sending.
  • Prefer narrow tools ("create draft reply") over broad ones ("run any API call").
  • Separate read and write. Scope by resource (this folder, this project).
  • Use short-lived credentials and rotate them.

4. Human approval for consequential actions

Define which actions need confirmation: external communications, payments, deletions, permission changes, production deployments. Show the user exactly what the agent intends to do. Approval fatigue is real, so reserve approvals for actions that matter.

5. Sandbox execution

Agents that run code or browse should do so in isolated environments with restricted network egress, no access to production secrets, and resource limits. Treat the sandbox as hostile.

6. Validate everything the agent passes on

Agent outputs that become inputs to other systems (SQL, shell, HTML, API calls) must be validated and encoded, just like user input. Don't let natural language become an execution path.

7. Secure the tool ecosystem

Agents increasingly connect to tools through protocols such as the Model Context Protocol (MCP), and to other agents through agent-to-agent (A2A) protocols. Each connection is a trust relationship:

  • vet tool servers like any third-party software: who maintains them, what permissions they request;
  • pin versions and review changes, since a tool's description can itself carry injected instructions;
  • authenticate agent-to-agent calls and don't let one agent's output become another's trusted instruction;
  • keep a registry of approved tools and agents.

8. Log and monitor

Record each agent run: the goal, the plan, the tools called with parameters, the data accessed and the results. Alert on anomalies such as bulk data reads, new external recipients, unusual hours, or repeated failed authorisation. Make it easy to stop an agent: a kill switch and a revocable token.

A quick threat-modelling exercise

For each agent, write down:

  1. Its goal and its users.
  2. Every tool and data source it can reach, and with whose authority.
  3. Every untrusted input it reads.
  4. The worst action it could take if an attacker controlled its instructions.
  5. The controls that stop or limit that action.

If step 4 is alarming and step 5 is thin, reduce the agent's privileges before launch.

Learn more

Agent security, AI threat modelling and AI governance are covered in our free AI Security & Governance course. For cloud identity foundations, see seven cloud identity mistakes.

Written and reviewed by the DigiTrust Asia editorial team. How we research, review and correct articles.

General information, not legal advice. Laws and official guidance change; check the current text before acting.

Build AI governance skills

AI Security & Governance is a free course with a final exam and certificate.

Create a free account