ShapedSI Journal · Applied AI
AI agents with human approval: design the decision before the action
An agent becomes easier to trust when its authority is easy to describe. Start by separating a suggestion from an action. The system can gather approved information and prepare a proposed change, while a person reviews the consequence before the agent sends, modifies or commits anything on the business’s behalf.
In this guide
01
Name the actions, not just the assistant
Write a list of the operations the agent could perform. Reading a service document, summarising an enquiry, drafting a response and sending that response are four separate operations. Give each one an owner and an explicit permission. Avoid describing the whole project as an assistant that will handle everything.
Decide which actions are reversible and which affect another person or system. A draft can be discarded; a sent message or changed record may require follow-up. The review point should reflect that difference. Start with the smallest useful set of capabilities and add another operation only when the team understands its consequence.
02
Make the approval understandable
Show the reviewer the proposed action, the affected records and the information used to prepare it. For a message, show the recipient and complete text. For a record update, show the current value and proposed value. The reviewer should not have to inspect a log of technical events to understand the decision.
Give the person a practical way to reject, edit or ask for more context. Define whether an approval applies to one operation or a clearly bounded batch. If the input changes after approval, the system should treat that as a new decision. Avoid collecting a broad approval that hides the details of later actions.
03
Handle uncertainty and external instructions
Information retrieved from an email, page or attachment may contain text asking the system to do something unrelated to the job. Treat that material as input to evaluate, rather than authority to expand the agent’s permissions. Keep the approved task and tool access separate from the material the agent reads.
Test examples with conflicting instructions, missing information and an unavailable tool. The agent should pause or route the work for review when it cannot complete the approved operation. Describe the fallback in ordinary language so the process owner knows what will happen during a difficult case.
04
Keep an accountable record
Record which operation was proposed, who approved it and what the system actually completed. Keep the information needed to investigate a mistake without storing unnecessary sensitive content. Distinguish a requested action from a successful action in the interface and in the record.
Review the workflow after a small set of real uses. Ask whether the approval screen gives enough context and whether staff are simply approving it from habit. Adjust the boundary where necessary. The goal is a useful decision that remains visible to the person responsible for the work.
Bring this to the conversation
A working brief.
- An explicit list of tool operations
- A named reviewer
- A clear before-and-after view
- A reject and edit path
- A record of approval and completion
A few useful questions.
Will approval remove every risk?
No. Approval is one control. The reviewer still needs useful context, limited permissions and a reliable way to check the result.
Can we approve a batch?
A bounded batch may be appropriate when the affected records and action are clear. Define the boundary and what changes require another review.
Original sources
Sources and further reading.
OpenAI’s Agent Builder safety guidance includes tool approvals and a human approval node. Its guidance also notes that input guardrails alone are not foolproof.
Shape the next step