Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 1 of 37

Agent

0 of 37 complete

Why it matters

Plenty of valuable tasks cannot be written down as a fixed sequence. "Find out why the nightly import fails", "add pagination to this endpoint", "reconcile these two invoices" all share a property: the number of steps and their order depend on what you discover along the way. A workflow has to encode that order in advance, so it only handles the cases the author imagined.

An agent moves that decision into the runtime. That is the whole idea, and it is also the source of every cost. Once the model chooses the path, you cannot know the token bill before the task starts, cannot guarantee the same path twice, and cannot test by enumerating branches. You can only bound the loop, constrain the tools and measure outcomes. Everything else on this track is a consequence of that trade.

Mental model

A contractor with a toolbox and a written brief, working in your house while you are away. You specify the outcome. They decide whether to measure first, cut, or ask for a part. Two things keep the arrangement safe: the toolbox contains only what you handed over, and a foreman stands between the contractor and the house, checking each action against the rules and calling a stop.

The analogy breaks in one important place. A contractor brings judgement from experience and knows when they are out of their depth. A model has neither durable judgement nor reliable self-knowledge: it will attempt a plausible action and then report that the work succeeded. The foreman is not optional, and the report is not evidence. That is why verification, not autonomy, is the hard part of agent engineering.

How it works

One run of the loop: read, propose, act, observe, decideInteractive
transitionfailurerepeat

Event · step 1 of 9

Run starts

Task plus constraints arrive together: what done means, read-only tools, 12 steps and 60k tokens. Nothing has been spent yet.

In state

  • Read

Next: Context assembled

Step
1/12
Tool calls
0
Transcript
1.2k
Step 1/9 · Run starts

Figure. Seven states and seven transitions, walked through one run that ends at the answer. The model proposes at Plan; Act, Observe and the stop checks are harness work. Answer is reached when the model returns no tool calls; the stop exit is the budget or repeat check.

What to notice. The model never moves the machine: it emits a proposal, and the harness validates, executes and records the result before the next decision. The loop back to Act is where cost accumulates, which is why both the step cap and the no-progress check attach to Decide. An answer is not a verified answer: the harness bounds the loop, it does not establish truth. Step counts, tool calls and token totals are illustrative.

Text version

A state machine with seven states. Read is the start, a pill. Read goes to Plan, a hexagonal decision state, on the event task plus tools. Plan goes to Act on dispatch. Act goes to Observe on the result. Observe goes to Decide on evidence. Decide has a dotted loop edge back to Act labelled repeat, a normal edge to Answer labelled no tool calls, and a red dashed failure edge to Stop labelled budget · no progress. Answer is a terminal pill; Stop is a red failure box. The trace runs Read, Plan, Act, Observe, Decide, Act again with a corrected query, Observe, Decide, Answer across nine steps. Counters show the step count out of twelve, tool calls and transcript tokens rising from 1.2k to 7.3k.

  1. Intake. The task arrives as a goal plus constraints: what done means, what is off-limits, what budget applies.
  2. Context assembly. The harness builds the first prompt: system instructions, task, relevant retrieved state, available tools.
  3. Decision. The model returns either a final answer or one or more tool calls with arguments.
  4. Dispatch. The harness looks up each tool by name, validates arguments against the schema, checks policy (may this caller run this tool on this resource), then executes.
  5. Observation. The result, including errors, is serialised back into the transcript as a tool message.
  6. Repeat. Steps 3 to 5 run until the model stops calling tools, a stop condition fires, or the budget is exhausted.
  7. Return. The harness records the outcome, the trajectory and the spend, and hands back a result plus an honest status.

Two details carry most of the engineering weight. The model never executes anything; it emits a structured request that the harness interprets. And the transcript, not a Python variable, is the state. If it is not in the message list or an explicit session store, the next turn cannot see it.

def run(task: str, tools: dict, budget: int = 12) -> str:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": task},
    ]
    for _ in range(budget):
        reply = model(messages, tools=[t.schema for t in tools.values()])
        messages.append(reply)
        if not reply.tool_calls:
            return reply.content
        for call in reply.tool_calls:
            tool = tools.get(call.name)
            if tool is None:
                result = {"error": f"unknown tool {call.name}"}
            else:
                try:
                    result = tool.run(**call.args)
                except ToolError as exc:
                    result = {"error": str(exc), "retryable": exc.retryable}
            messages.append({"role": "tool", "id": call.id, "content": json.dumps(result)})
    raise BudgetExceeded(task)

Read the loop for what it does not do. It does not deduplicate actions, persist progress, cap token growth, or confirm that the final answer is true. Those omissions are the production work described in agent loop and stop conditions.

Control flow
Model decides at runtime
Executes tools
Harness, never the model
State
Transcript plus session store
Default budget
Hard step and token caps
Failure shape
Plausible, unverified success

Trade-offs

DesignGood forCostFails when
Single model callClassification, extraction, draftingNo tool access, no recoveryThe answer depends on state the model cannot see
Fixed workflowKnown, repeatable procedureBreaks on unanticipated casesInputs vary more than the author assumed
Agent with toolsOpen-ended tasks with verifiable stepsVariance in cost and latency; needs tracesNo cheap way to check whether the result is right
Agent plus human gateIrreversible actions, regulated flowsHuman latency and attentionApprovals arrive faster than anyone can read them
Multi-agent systemGenuinely separable subtasksCoordination overhead, harder debuggingThe task is sequential
Trade-off mapInteractive

Good for

Classification, extraction, drafting

Cost

No tool access, no recovery

Fails when

The answer depends on state the model cannot see

  1. The answer depends on state the model cannot see
  2. Inputs vary more than the author assumed
  3. No cheap way to check whether the result is right
  4. Approvals arrive faster than anyone can read them
  5. The task is sequential

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Single model call is good for Classification, extraction, drafting, costs No tool access, no recovery, and fails when The answer depends on state the model cannot see. Fixed workflow is good for Known, repeatable procedure, costs Breaks on unanticipated cases, and fails when Inputs vary more than the author assumed. Agent with tools is good for Open-ended tasks with verifiable steps, costs Variance in cost and latency; needs traces, and fails when No cheap way to check whether the result is right. Agent plus human gate is good for Irreversible actions, regulated flows, costs Human latency and attention, and fails when Approvals arrive faster than anyone can read them. Multi-agent system is good for Genuinely separable subtasks, costs Coordination overhead, harder debugging, and fails when The task is sequential

Failure modes

  • Unverified success. The agent reports completion without running anything that could contradict it. Debug: require a verification tool call before the final answer and check the trace for it; if the trace lacks it, treat the run as failed regardless of the text.
  • Repeating a losing action. The same call with the same arguments recurs because the observation did not change the model's mind. Debug: hash tool name plus arguments per step; three identical hashes is a loop, so abort with the transcript attached.
  • Silent context overflow. The transcript grows past the window and the harness truncates the oldest messages, which may include the task constraints. Debug: log prompt tokens per step and a hash of the retained system prompt; alert when truncation first occurs.
  • Permission by accident. A tool is registered globally and becomes reachable from a context that should not have it. Debug: diff the registry between releases and assert the allowed tool set per route at startup.

In production

Cost is long-tailed. A median run of a few steps and a tail an order of magnitude longer is the normal shape, so a per-task budget cap matters more than an average estimate. Cap steps, tokens and wall-clock separately, because one can blow up while the others look fine.

Monitor steps per task, tool error rate, budget-exhaustion rate, escalation rate and cost per resolved task. Then add the metric most teams skip: verified success rate, meaning runs where an independent check confirmed the outcome. Raw success rate rewards agents that declare victory. Quality, not volume, is the signal worth alerting on; see agent evaluation.

Common misconceptions

People think: An agent is any LLM application that can call tools.

Actually: Tool calling is a capability. An agent is a control pattern where the model chooses the sequence of actions. A tool-using chatbot that always answers in one turn is not an agent.

People think: More autonomy means a more capable system.

Actually: Autonomy is a cost, not a feature. Every removed constraint is a path you can no longer predict, budget or test. The useful question is which decisions genuinely need runtime judgement.

Interview angle

The question behind the question is usually "can you tell an agent from a workflow, and do you know why the distinction matters?" Define the agent by control-flow ownership, then immediately name the price: non-deterministic cost, trajectory-level evaluation, and the need for verification the model cannot fake. Interviewers listen for whether you bound the loop before describing the tools. See agents interview questions and the architecture sketch in reference architecture: agent.

Sources