Answer in brief
Start an AI agent with read-only access and its own account, then add one permission at a time. Sort its actions into three classes: reading and drafting without approval, reversible internal changes with review, and irreversible or external actions approved one by one by a named person. Enforce the rules in the systems themselves, cap volume and spend, keep a log the agent cannot edit and test the stop procedure before launch.
A mistake costs what the permissions allow
An AI agent is software allowed to act: read a mailbox, update a record, send a message, issue a refund. The model inside it will sometimes be wrong, and sometimes be misled by text it reads along the way. Neither can be ruled out by better wording. What can be decided in advance is how far a wrong step can travel.
The OWASP Gen AI Security Project names the risk Excessive Agency: damaging actions performed in response to unexpected, ambiguous or manipulated outputs from a language model. It traces the root cause to one or more of three things: excessive functionality, permissions or autonomy. All three are choices made by whoever sets the agent up.
That is why it helps to treat AI agent permissions as something earned. The agent begins with almost none, shows over a period of logged work that a class of action is safe in its hands, and only then receives the next.
List the actions before you connect the tools
Write the task as a sequence of steps and, for each, note what the agent must read and what it must change. The list is usually shorter than the tool’s feature set. OWASP’s example is a mailbox assistant that summarises incoming email: it needs to read messages, so the extension it uses should not also be able to delete or send them.
Then remove what the list does not justify. OWASP’s first recommendations are to minimise the extensions an agent may call, to minimise the functions inside each one and to avoid open-ended tools, such as running a shell command or fetching any URL, in favour of narrow ones. A tool tried during development and later abandoned should be disconnected, not merely ignored.
Apply the same restraint to data. In OWASP’s illustration, an agent that recommends products needs read access to the products table only, with no ability to insert, update or delete records or to see other tables. For a small business the equivalent is one shared folder or one CRM pipeline, not the whole account.
Read-only comes first, and it lasts for weeks
For the first period the agent reads, searches and drafts, and a person carries out every action. OpenAI’s guide to building agents distinguishes data tools, which retrieve information, from action tools, which change something in another system. Begin with the first kind only. The agent’s output is a proposal: a drafted reply, a suggested update, a list of invoices that look wrong.
This stage is not wasted time. It shows how often the drafts are accepted unchanged, which cases the agent gets wrong and whether its mistakes are small or strange. The same guide calls human intervention especially important early in deployment, because it helps to identify failures and uncover edge cases.
Decide the promotion rule before you start, so impatience cannot bend it later. For example: one category of action becomes automatic only after an agreed number of consecutive drafts were approved without edits, and it returns to manual after any serious error. The numbers are yours to set; what matters is that they were written down first.
Who approves what: three classes of action
Sort every listed action into one of three classes. The first is reading, searching and drafting: no approval, but logged. The second is a change inside your own systems that can be undone, such as tagging a ticket, filling a field that keeps its history or saving a draft. The process owner can approve these in batches or check them by sampling.
The third class is anything irreversible, external or financial: messaging a customer, publishing, deleting, paying, refunding, changing access rights. OWASP recommends human-in-the-loop control, so that a person approves high-impact actions before they are taken. OpenAI’s guide says sensitive, irreversible or high-stakes actions should trigger human oversight until confidence in the agent’s reliability grows, and gives cancelling orders, authorising large refunds and making payments as examples.
Name the approver by role, and name a deputy, because an approval queue with nobody in it becomes a reason to switch the control off. Show the approver the exact action, the data it rests on and what happens next, not a summary the agent wrote about itself.
The rule lives in the system, not in the prompt
An instruction such as “never delete records” is a request to the model. It is not a control, because the model is the part that may fail. OWASP’s advice is complete mediation: implement authorisation in the downstream systems and do not rely on a language model to decide whether an action is allowed.
In practice the agent gets its own account, not a staff login and never an administrator’s key, with the narrowest role the system offers. Where the agent acts for a particular user, OWASP recommends executing in that user’s context with the minimum privileges necessary, for example through an OAuth authorisation with the minimum scope required.
The approval step belongs in the same place. If refunds above a threshold need a second signature in the payment system, the agent meets that rule as an employee would. OpenAI’s guide adds that guardrails should be coupled with robust authentication and authorisation, strict access controls and standard software security measures.
Limits on volume, spend and reach
A limit turns an open-ended failure into a bounded one. OWASP notes that rate limiting reduces the number of undesirable actions that can take place in a given period and increases the chance of discovering them through monitoring before significant damage occurs. One wrong email is an apology; a thousand are an incident.
Set the limits where the agent cannot change them: a maximum number of actions per run and per day, a cap on recipients, a list of permitted folders, domains or record types. Anthropic’s guidance on building agents mentions stopping conditions, such as a maximum number of iterations, as a common way to maintain control.
Cost needs a ceiling too. Anthropic observes that the autonomous nature of agents means higher costs and the potential for compounding errors. Where the model provider’s account offers a spending limit or an alert, set it below the figure that would hurt. OpenAI’s guide adds a behavioural limit: cap the agent’s retries and hand the case to a person when the cap is reached.
A log somebody actually reads
Record every tool call: the time, the task, the tool, its input, the result and who approved it. Keep it where the agent has no permission to edit. OWASP is frank about what this achieves: logging and monitoring will not prevent excessive agency, but they limit the damage by showing where undesirable actions are taking place.
A log protects nobody until someone reads it. Give one named owner a weekly review with three questions: what did the agent do that surprised us, what did approvers reject and why, and which limits were reached. The answers decide whether a permission is promoted, kept or withdrawn.
NIST’s voluntary AI Risk Management Framework states the same expectations as outcomes. One subcategory asks that the behaviour of an AI system be monitored in production. Another asks that incidents and errors be communicated to the relevant people, including affected communities, and that processes for tracking, responding to and recovering from them be followed and documented.
The way back is written before launch
The same framework asks for mechanisms, with responsibilities assigned and understood, to supersede, disengage or deactivate an AI system whose performance or outcomes are inconsistent with its intended use. For a small team that is one sentence: who may switch the agent off, and how. That person should be able to do it alone, by disabling the integration or revoking the agent’s credentials, without waiting for a developer.
Then write the undo path for each reversible action. A changed record is restored from its history, a moved file from the previous folder, a draft is simply discarded. Take a backup or an export before any bulk operation. If an action has no undo path, it was misfiled: it belongs in the third class and waits for approval.
Rehearse it once on copied data. Stop the agent in the middle of a run, restore one change and time both. Decide who tells customers if something reached them in error. The moment to find the gaps in a plan is before the first live run.
One page to settle before the first live run
Before launch, put the decisions on one page. State the purpose of the agent and the tasks it must not attempt. List its actions by class, with the approver and deputy for each. Record which account it uses, with what role, and when its credentials expire. Add the limits, where the log is kept, who reviews it, the stop procedure and the date of the rehearsal.
Test against that page in a sandbox first, on copies of real data. Anthropic recommends extensive testing in sandboxed environments together with appropriate guardrails. Include hostile cases: a document containing instructions addressed to the agent, a request outside its purpose, a tool that returns an error. Note what it did, not what it said it would do.
NIST’s framework asks that roles, responsibilities and lines of communication be documented and clear, and that processes for human oversight be defined, assessed and documented. NIST calls the framework a living document, so check its site for the current version. Give your own page a review date too. AI agent permissions are added one at a time, and each should still have an owner a year from now.
Practical checklist
- List every action the agent could take and disconnect the tools and functions the task does not need.
- Give the agent its own account with read-only access first, and add one permission at a time.
- Name the person who approves each irreversible, external or financial action, and a deputy for absences.
- Set limits on actions per run, recipients and daily spend in the systems themselves, not in the prompt.
- Rehearse the stop procedure and one rollback on copied data before the agent touches live work.
Questions and answers
What does least privilege mean for an AI agent?
It means the agent has only the tools, functions and data access that its defined task requires, and nothing that is merely convenient. OWASP’s examples are concrete: an assistant that summarises email should be able to read messages but not send or delete them, and an extension that reads a database should connect with an identity limited to reading. The test is whether each permission can be traced to a step of the task.
Which AI agent actions should always need human approval?
Those that cannot be undone, that leave your organisation or that move money or access: sending to customers, publishing, deleting, paying, refunding and changing permissions. OpenAI’s guide gives order cancellations, large refunds and payments as examples; OWASP adds posting to social media on a user’s behalf. Approval can be relaxed later for a narrow category, but only by a decision that is recorded.
Is an instruction in the prompt enough to stop an agent doing something?
No. A prompt guides the model, and the model is the component that can be confused or manipulated. OWASP recommends enforcing authorisation in the systems the agent connects to, so that a forbidden request is refused there whatever the model decides. Use the prompt to explain the job, and use roles, scopes and approval rules in the target systems to set the boundaries.
When is it safe to give an agent more permissions?
When there is evidence and a way back. Evidence means a logged period in which the agent’s drafts for that category were reviewed and accepted against a threshold you set in advance. A way back means you know how to undo that kind of action and how to withdraw the permission. Add one permission at a time, so that a new problem can be traced to a single change.
Does a small business need a formal framework to manage an AI agent?
Not a formal programme. NIST’s AI Risk Management Framework is voluntary and written for organisations of all sizes, and its questions scale down well: who is responsible, how is human oversight defined, how is the system monitored, and how is it switched off. A single page that answers them, kept up to date, does the essential work for a small team.