Why it matters
Agents can act, and some actions cannot be undone: money moves, emails leave, records are deleted, infrastructure is torn down. Least privilege reduces what the agent can do; approval decides what it may do now, with a person accountable for the specific action.
The reason to build this properly is that a badly built approval gate is worse than none. It produces a paper trail that suggests oversight while the person clicks through a summary they did not read. When the incident review asks who authorised the transfer, "the approver" is not an answer if the approver saw a different amount.
Mental model
A co-signature on a cheque. The signer sees the amount, the payee and the date, and signs that instrument. They do not sign a blank cheque and let a clerk fill it in later.
The analogy breaks on volume and comprehension. A bank signer handles a few instruments a day and understands each. An approver in an agent system may handle dozens of requests in an hour, each generated by a system they did not write, described in prose the agent wrote. The gate has to be designed for that attention budget, which means showing the diff, not the narrative.
How it works
- Classify the action. Assign a risk tier from reversibility, blast radius and external visibility. Reading a file is tier 0. Sending an internal message is tier 1. Moving money or deleting data is tier 3.
- Pause before execution, not after. The agent produces a proposed action: tool name, resolved arguments, target resource, and the intended effect in one sentence.
- Freeze the payload. Compute a digest over the canonical action and store it with the request. Approval applies to that digest only.
- Show the diff, not the plan. The approver needs the concrete change: the amount and destination, the file and hunk, the recipient list and body. A link to the trace is a supplement, never the primary view.
- Record the decision with identity and time. Who approved, when, from where, and what they were shown. Store the rendered view or its hash alongside the payload hash.
- Execute exactly the approved payload. If the agent wants to change a parameter after approval, that is a new request, not an edit.
- Expire and consume. Approvals have a short lifetime and are single-use. Expiry is what stops a queue of old yeses from becoming a permission.
- Fail closed. If the approval service is unavailable, the action does not run. Degrading to auto-approval is not a degraded mode, it is a different system.
from dataclasses import dataclass
from datetime import datetime, timedelta
import hashlib, json
@dataclass(frozen=True)
class PendingAction:
id: str
tool: str
args: dict
requested_by: str
risk_tier: int
def digest(self) -> str:
body = json.dumps({"tool": self.tool, "args": self.args}, sort_keys=True)
return hashlib.sha256(body.encode()).hexdigest()
@dataclass(frozen=True)
class Approval:
action_id: str
digest: str
approver: str
approved_at: datetime
expires_at: datetime
single_use: bool = True
def execute_if_approved(action: PendingAction, approval: Approval | None, now: datetime):
if approval is None:
raise Blocked("no approval")
if approval.digest != action.digest():
raise Blocked("payload changed after approval")
if now > approval.expires_at:
raise Blocked("approval expired")
if approval.approver == action.requested_by:
raise Blocked("self-approval is not approval")
# Atomic compare-and-set in the approval store: first caller wins,
# every retry or concurrent duplicate gets False.
if not claim_once(approval.id):
raise Blocked("approval already consumed")
return dispatch(action.tool, action.args)Four checks, four different incidents prevented: missing approval, changed payload, stale approval, self-approval. The atomic claim is the fifth, and its absence is the one that shows up in postmortems as a duplicated transfer. A check-then-consume sequence is not enough; two concurrent requests can both pass the check.
Keyed transitions
- Verify → Execute: digest ∧ fresh ∧ approver ≠ requester ∧ claim
Event · step 1 of 8
Proposed
Entry: the agent proposes a 480 refund with resolved arguments and the order id. Nothing is dispatched until a decision is bound to this exact payload.
In state
- Propose
Next: Frozen
- Transfers
- 0
- Approval age
- —
Figure. Nine states. Freeze canonicalises the action and hashes it before anyone is asked. Pending is durable, so the wait costs no worker; Verify rechecks the digest, the clock, the approver and the single-use claim before Execute can run. Mismatch, Rejected and Expired are the three ways an approval ends without a second transfer.
What to notice. The digest is what binds the yes to the amount the approver saw; the atomic claim is what stops a redelivered approval from executing twice. Execute runs the frozen payload, so a parameter change after approval is a new request. A rejection is a valid decision and ends at a terminal state, not at the failure tone used for expiry. The 500 threshold, 480 refund and 15-minute expiry are illustrative, and the approval UI, queue staffing and audit retention are omitted.
Text version
A state machine with nine states. Propose goes to Freeze on resolved arguments; Freeze goes to Pending when the tier is at least 3, and to Execute when it is below the gate. Pending goes to Verify on approved, to Rejected on reject, and to Expired on expiry. Verify goes to Execute when the digest matches, the approval is fresh, the approver is not the requester and the claim succeeds, and to Mismatch when any guard fails. Execute goes to Audit on a recorded decision. The trace walks Propose, Freeze, Pending, Verify, Execute, Audit, then Mismatch on a reused approval, then Expired on expiry. Counters show one transfer and the approval age.
Trade-offs
| Design | Good for | Cost | Fails when |
|---|---|---|---|
| No gate, least privilege only | Read-only agents, sandboxes | No human judgement on edge cases | A permitted action is still harmful in context |
| Approve every write | Early pilots, low volume | Approval fatigue within days | Volume exceeds reviewer attention |
| Risk-tiered gates | Most production systems | Requires a maintained policy | Tiers are set once and never revisited |
| Two-person approval | High-value, regulated actions | Latency and staffing | Both approvers are the same on-call person |
| Post-hoc review with rollback | Reversible, high-volume actions | Damage lands before review | Rollback is impossible or incomplete |
Failure modes
- Rubber-stamping. Median decision time collapses to under a second and approvals track request volume one-to-one. Debug: measure time-to-decision and approval rate per approver; a rate near 100% with sub-second decisions means the gate is decorative.
- Approval drift. The agent modifies an argument after approval and the action still runs. Debug: assert payload digest equality at execution and count mismatches; any non-zero count is a live vulnerability.
- Context-free approval. The request says "update the customer record" without the field values. Debug: sample ten approved requests and ask a reviewer to state what changed; if they cannot, redesign the view.
- Self-approval by another name. The agent's service account is also an approver, or the requesting user approves their own request. Debug: join requester and approver identities in the audit log and alert on equality.
In production
Approval adds human latency to a machine-speed loop, so it belongs on the exception path. Design for a queue: what happens when nobody approves for an hour, how the request is cancelled, whether the agent holds a lock meanwhile. Most incidents here are about held resources and duplicated work, not about the decision itself.
Audit records should be immutable and complete enough to reconstruct the screen the approver saw. For regulated flows, that record is the control. Also measure the gate's effect: how many actions were blocked, how many were modified before approval, and how often approval was bypassed through another path. A gate that never blocks anything is either perfectly scoped or not working.
Common misconceptions
People think: Adding an approval step makes the system safe.
Actually: It makes one class of action reviewable. Safety depends on what the approver can see, whether the payload is bound, and whether the volume permits attention.
People think: Approval is a compliance feature, so it can be added before launch.
Actually: Approval changes the execution path, the state model and the failure behaviour. It is architecture, and retrofitting it into a loop with no pause or resume is a rewrite.
Interview angle
The question is usually framed as a design constraint: "the agent can issue refunds, what stops it from issuing the wrong one?" Strong answers separate capability from authorisation, bind the approval to a payload digest, and name the expiring single-use property. Then discuss the human factors: what the approver sees and how volume erodes attention. See agents interview questions, least privilege and interrupts and resume.
Sources
- NIST, AI Risk Management Framework — governance framing for human oversight proportional to risk.
- Microsoft, HAX Toolkit guidelines — guidance on meaningful human control over AI actions.
- Google PAIR, People + AI Guidebook: Errors and Graceful Failure — designing recovery paths that keep the person informed.
- OWASP, Top 10 for LLM Applications — excessive agency as a named risk class, with mitigation guidance.