Why it matters
Plenty of valuable tasks cannot be written down as a fixed sequence. "Find out why the nightly import fails", "add pagination to this endpoint", "reconcile these two invoices" all share a property: the number of steps and their order depend on what you discover along the way. A workflow has to encode that order in advance, so it only handles the cases the author imagined.
An agent moves that decision into the runtime. That is the whole idea, and it is also the source of every cost. Once the model chooses the path, you cannot know the token bill before the task starts, cannot guarantee the same path twice, and cannot test by enumerating branches. You can only bound the loop, constrain the tools and measure outcomes. Everything else on this track is a consequence of that trade.
Mental model
A contractor with a toolbox and a written brief, working in your house while you are away. You specify the outcome. They decide whether to measure first, cut, or ask for a part. Two things keep the arrangement safe: the toolbox contains only what you handed over, and a foreman stands between the contractor and the house, checking each action against the rules and calling a stop.
The analogy breaks in one important place. A contractor brings judgement from experience and knows when they are out of their depth. A model has neither durable judgement nor reliable self-knowledge: it will attempt a plausible action and then report that the work succeeded. The foreman is not optional, and the report is not evidence. That is why verification, not autonomy, is the hard part of agent engineering.
How it works
Event · step 1 of 9
Run starts
Task plus constraints arrive together: what done means, read-only tools, 12 steps and 60k tokens. Nothing has been spent yet.
In state
- Read
Next: Context assembled
- Step
- 1/12
- Tool calls
- 0
- Transcript
- 1.2k
Figure. Seven states and seven transitions, walked through one run that ends at the answer. The model proposes at Plan; Act, Observe and the stop checks are harness work. Answer is reached when the model returns no tool calls; the stop exit is the budget or repeat check.
What to notice. The model never moves the machine: it emits a proposal, and the harness validates, executes and records the result before the next decision. The loop back to Act is where cost accumulates, which is why both the step cap and the no-progress check attach to Decide. An answer is not a verified answer: the harness bounds the loop, it does not establish truth. Step counts, tool calls and token totals are illustrative.
Text version
A state machine with seven states. Read is the start, a pill. Read goes to Plan, a hexagonal decision state, on the event task plus tools. Plan goes to Act on dispatch. Act goes to Observe on the result. Observe goes to Decide on evidence. Decide has a dotted loop edge back to Act labelled repeat, a normal edge to Answer labelled no tool calls, and a red dashed failure edge to Stop labelled budget · no progress. Answer is a terminal pill; Stop is a red failure box. The trace runs Read, Plan, Act, Observe, Decide, Act again with a corrected query, Observe, Decide, Answer across nine steps. Counters show the step count out of twelve, tool calls and transcript tokens rising from 1.2k to 7.3k.
- Intake. The task arrives as a goal plus constraints: what done means, what is off-limits, what budget applies.
- Context assembly. The harness builds the first prompt: system instructions, task, relevant retrieved state, available tools.
- Decision. The model returns either a final answer or one or more tool calls with arguments.
- Dispatch. The harness looks up each tool by name, validates arguments against the schema, checks policy (may this caller run this tool on this resource), then executes.
- Observation. The result, including errors, is serialised back into the transcript as a tool message.
- Repeat. Steps 3 to 5 run until the model stops calling tools, a stop condition fires, or the budget is exhausted.
- Return. The harness records the outcome, the trajectory and the spend, and hands back a result plus an honest status.
Two details carry most of the engineering weight. The model never executes anything; it emits a structured request that the harness interprets. And the transcript, not a Python variable, is the state. If it is not in the message list or an explicit session store, the next turn cannot see it.
def run(task: str, tools: dict, budget: int = 12) -> str:
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": task},
]
for _ in range(budget):
reply = model(messages, tools=[t.schema for t in tools.values()])
messages.append(reply)
if not reply.tool_calls:
return reply.content
for call in reply.tool_calls:
tool = tools.get(call.name)
if tool is None:
result = {"error": f"unknown tool {call.name}"}
else:
try:
result = tool.run(**call.args)
except ToolError as exc:
result = {"error": str(exc), "retryable": exc.retryable}
messages.append({"role": "tool", "id": call.id, "content": json.dumps(result)})
raise BudgetExceeded(task)Read the loop for what it does not do. It does not deduplicate actions, persist progress, cap token growth, or confirm that the final answer is true. Those omissions are the production work described in agent loop and stop conditions.
- Control flow
- Model decides at runtime
- Executes tools
- Harness, never the model
- State
- Transcript plus session store
- Default budget
- Hard step and token caps
- Failure shape
- Plausible, unverified success
Trade-offs
| Design | Good for | Cost | Fails when |
|---|---|---|---|
| Single model call | Classification, extraction, drafting | No tool access, no recovery | The answer depends on state the model cannot see |
| Fixed workflow | Known, repeatable procedure | Breaks on unanticipated cases | Inputs vary more than the author assumed |
| Agent with tools | Open-ended tasks with verifiable steps | Variance in cost and latency; needs traces | No cheap way to check whether the result is right |
| Agent plus human gate | Irreversible actions, regulated flows | Human latency and attention | Approvals arrive faster than anyone can read them |
| Multi-agent system | Genuinely separable subtasks | Coordination overhead, harder debugging | The task is sequential |
Good for
Classification, extraction, drafting
Cost
No tool access, no recovery
Fails when
The answer depends on state the model cannot see
- The answer depends on state the model cannot see
- Inputs vary more than the author assumed
- No cheap way to check whether the result is right
- Approvals arrive faster than anyone can read them
- The task is sequential
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Single model call is good for Classification, extraction, drafting, costs No tool access, no recovery, and fails when The answer depends on state the model cannot see. Fixed workflow is good for Known, repeatable procedure, costs Breaks on unanticipated cases, and fails when Inputs vary more than the author assumed. Agent with tools is good for Open-ended tasks with verifiable steps, costs Variance in cost and latency; needs traces, and fails when No cheap way to check whether the result is right. Agent plus human gate is good for Irreversible actions, regulated flows, costs Human latency and attention, and fails when Approvals arrive faster than anyone can read them. Multi-agent system is good for Genuinely separable subtasks, costs Coordination overhead, harder debugging, and fails when The task is sequential
Failure modes
- Unverified success. The agent reports completion without running anything that could contradict it. Debug: require a verification tool call before the final answer and check the trace for it; if the trace lacks it, treat the run as failed regardless of the text.
- Repeating a losing action. The same call with the same arguments recurs because the observation did not change the model's mind. Debug: hash tool name plus arguments per step; three identical hashes is a loop, so abort with the transcript attached.
- Silent context overflow. The transcript grows past the window and the harness truncates the oldest messages, which may include the task constraints. Debug: log prompt tokens per step and a hash of the retained system prompt; alert when truncation first occurs.
- Permission by accident. A tool is registered globally and becomes reachable from a context that should not have it. Debug: diff the registry between releases and assert the allowed tool set per route at startup.
In production
Cost is long-tailed. A median run of a few steps and a tail an order of magnitude longer is the normal shape, so a per-task budget cap matters more than an average estimate. Cap steps, tokens and wall-clock separately, because one can blow up while the others look fine.
Monitor steps per task, tool error rate, budget-exhaustion rate, escalation rate and cost per resolved task. Then add the metric most teams skip: verified success rate, meaning runs where an independent check confirmed the outcome. Raw success rate rewards agents that declare victory. Quality, not volume, is the signal worth alerting on; see agent evaluation.
Common misconceptions
People think: An agent is any LLM application that can call tools.
Actually: Tool calling is a capability. An agent is a control pattern where the model chooses the sequence of actions. A tool-using chatbot that always answers in one turn is not an agent.
People think: More autonomy means a more capable system.
Actually: Autonomy is a cost, not a feature. Every removed constraint is a path you can no longer predict, budget or test. The useful question is which decisions genuinely need runtime judgement.
Interview angle
The question behind the question is usually "can you tell an agent from a workflow, and do you know why the distinction matters?" Define the agent by control-flow ownership, then immediately name the price: non-deterministic cost, trajectory-level evaluation, and the need for verification the model cannot fake. Interviewers listen for whether you bound the loop before describing the tools. See agents interview questions and the architecture sketch in reference architecture: agent.
Sources
- Anthropic, Building effective agents — distinguishes workflows from agents by who controls the path, and argues for the simplest pattern that works.
- Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models — interleaves reasoning traces with tool actions, the loop shape most agents still use.
- Sumers et al., Cognitive Architectures for Language Agents — frames agents as memory, decision procedure and action space, which is a useful vocabulary for design reviews.
- Russell & Norvig, Artificial Intelligence: A Modern Approach — the classical agent-environment framing that predates LLMs and still describes the loop.