Guardrails before an AI agent runs

Read, change, refuse

AI guardrails come first. An agent that can call a tool can change a record, so sit with the person who owns the process and write three limits before that call is allowed. The system may read some data. It may change some records. It must refuse the rest.

In the invoice process used across these guides, the three lists might read like this. The agent may read incoming invoices, purchase orders, and the supplier list. It may set an invoice’s status to matched or flagged and attach a note. It must refuse to change supplier bank details, release a payment, or act on any instruction that arrives inside an invoice.

Writing the limits is half the job. The other half is where they live. A sentence in the prompt asking the agent not to touch bank details is a request. A credential that has no permission to touch bank details is a limit. OpenAI’s guide of 15 January 2026, A practical guide to building agents, says the same thing: model guardrails should be paired with authentication, authorisation, strict access control, and ordinary software security. Give the agent its own account, scoped to the three lists, so that a mistake or a manipulated instruction runs into a wall rather than a polite reminder.

The security community has arrived at the same priority. The OWASP GenAI Security Project list of the most serious risks in LLM applications, in its 2026 edition published in August, moved “excessive agency” from sixth place to third. OWASP traces it to three root causes, and each maps to one of the lists:

• Excessive functionality. A tool that does more than the job needs, such as a document reader that can also delete. Fix it in the tool itself.
• Excessive permissions. A narrow feature running on an account that can do far more. Fix it in the account.
• Excessive autonomy. No person between the agent and an action that cannot be undone. Fix it in the handoff, covered in Where a person stays on the path.

The method around this is in How to build an AI agent that holds. How we take one measured process into production sits on the AI agents page.

Layers on the same run

OpenAI describes guardrails as layers. No single one is enough, and together they make the system harder to break. Their list:

• Relevance: flags requests outside the job, so the agent stays on its task.
• Safety: detects jailbreaks and prompt injections, meaning attempts to override the instructions or extract them.
• Personal data: checks output before a personal detail leaves the system unnecessarily.
• Moderation: flags harmful or abusive input.
• Tool safeguards: a risk rating on each tool, covered below.
• Plain rules: blocklists, length limits, and pattern matching, also covered below.
• Output validation: checks that what goes out matches the organisation’s standards.

Their advice on building the stack is practical. Start with data privacy and content safety. Add a new layer each time a real case gets through. Tune for both security and usability, because a guardrail that blocks normal work will get switched off.

One detail in the guide deserves more attention than it gets. In OpenAI’s Agents SDK, guardrails run alongside the agent by default rather than before it, and stop the run if they trip. That is fine for a reply, which can be discarded. It is not fine for a tool that writes: by the time a parallel check trips, the record may already have changed. For any action tool, the check has to finish before the call.

Classifiers lower the risk of prompt injection, but they do not close it. Anthropic’s note of April 2026, Trustworthy agents in practice, describes layered defences against it, from training to monitoring to red-teaming. It still says that together they are not a guarantee, and it asks builders to think hard about which tools, data, and permissions an agent gets.

Simon Willison’s lethal trifecta, from June 2025, turns that into a design rule. An agent becomes easy to steal from when it has all three of: access to private data, exposure to content an attacker can control, and a way to send information out. The reliable defence is not a better filter. It is removing one of the three.

The invoice process has all three by default. The finance system is private data. Invoices arrive from outside, and anyone can write text into a PDF. If the agent can also email whoever it likes, a single crafted invoice could ask it to send the supplier list somewhere else. So we take a leg away. The agent sends no email at all, or replies only to the address already on file, through a tool that does not accept a recipient as an argument.

A risk rating on each tool

OpenAI rates each tool low, medium, or high before it runs. The factors they name are whether the tool reads or writes, whether its action can be undone, which permissions it needs, and whether money is involved. The rating then decides what happens: a high-risk call can pause for an extra check or go to a person.

The person who owns the process sets each rating with us. For the invoice process:

Tool	Reads or writes	Can be undone	Money	Rating	What happens
Read invoice	Reads	Not applicable	No	Low	Runs
Look up purchase order	Reads	Not applicable	No	Low	Runs
Set status to matched or flagged	Writes	Yes	No	Medium	Runs, logged, checked in code first
Queue for payment	Writes	Until the payment run	Yes	High	Waits for a person
Change supplier bank details	Writes	Hard to detect	Yes	Not given	The agent has no such tool

The last row is the strongest guardrail in the table. The safest way to manage a high-risk tool is often not to give it to the agent at all. Review the ratings whenever a tool changes. A tool that only read yesterday and writes today is a new tool.

The check before the next step

A wrong step becomes the input of the next one, so errors compound. Anthropic’s engineering note of 19 December 2024, Building effective agents, is where that warning is sharpest. Its answer is for the agent to get ground truth from the environment at each step, not to rely on its own account.

In practice, put a check between steps, in code wherever possible. In the invoice process, after the agent reads an invoice and before anything is matched, check that the line items add up to the total, that the order number exists, and that the supplier on the invoice is the supplier on the order. A case that fails goes to a person with the reason attached. It does not go to the next step.

The check on the output happens before the next step. A model that has decided it is finished is not that check.

Plain rules next to the model

OpenAI also names rules that do not need a model: a blocklist, a length limit, a pattern a machine can match. We set those with the same person who owns the process. They are cheap, they give the same answer every time, and they are easy to test. They cover the cases you can already name. The model-based layers cover the ones you can’t.

Every guardrail is also a test case. Add cases to the evaluation set that should trip each guardrail and cases that should not. A guardrail that never fires has not been shown to work, and one that fires on normal work will be removed by the people it slows down. Log every trip. That log is where the next layer comes from.

The questions are the same at the end as at the start. What may the system read? What may it change? What must it refuse? And is each answer enforced somewhere the model cannot argue with? Then see the method, in How to build an AI agent that holds.