Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 2 of 37

Agent vs chatbot vs workflow

0 of 37 complete

Why it matters

The words get used as synonyms in marketing and as tribes in engineering debates, which hides a concrete architectural decision. If you cannot say which of the three shapes you are building, you cannot say what a test looks like, what a per-request cost bound is, or what "correct" means when it fails.

The distinction also decides where the interesting bugs live. Workflows fail at the edge of their branch conditions. Chatbots fail at retrieval and grounding. Agents fail in the gap between doing something and knowing it worked.

Mental model

Think of a train, a taxi and a receptionist. A train follows rails you laid: fast, predictable, useless when the destination is off the line. A taxi takes a route chosen at runtime: it reaches places the rails do not, and the fare depends on traffic. A receptionist mostly answers questions and hands out forms, occasionally walking you to a room.

The analogy breaks on repeatability. A taxi driver converges on a good route through experience; an agent does not accumulate skill between runs. Run the same task twice and you may get two different routes with two different bills, and neither driver remembers the last trip unless you store it yourself.

How it works

Classify by the guard, not by the labelInteractive

Keyed transitions

  1. Cheap check? → Hybrid: partly verifiable
transitionfailurerepeat

Event · step 1 of 8

Task classified

Entry: a support flow that issues refunds. The class is read from what the task needs, before a model, framework or tool count is chosen.

In state

  • Task

Next: Fixed steps?

Step 1/8 · Task classified

Figure. Nine states. Three guards decide the shape: whether the steps can be written down, whether the next step depends on values only execution reveals, and whether each step can be checked cheaply. Two failure states sit outside the good outcomes: a fixed sequence implemented as an open loop, and an open loop shipped with no cheap check. The hybrid is the usual answer when only part of the task can be checked cheaply.

What to notice. The guards describe the task, not the model or the tool count. A workflow with one open node is the usual answer when only part of the task is runtime-dependent; the failure states are what the label hides. The trace is one worked example of the guards, not a survey of systems; cost classes are qualitative, and the figure omits routing, retrieval and multi-agent delegation.

Text version

A state machine with nine states. Task goes to Fixed steps on its class; Fixed steps goes to Workflow on yes and to Runtime values on no. Runtime values goes to Workflow on no and to Cheap check on yes. Cheap check goes to Agent when every step is verifiable, to Hybrid when only part of it can be checked cheaply, and to Open loop when it is shipped without a check. Workflow goes to Costume when it is implemented as an open loop. The trace walks Task, Fixed steps, Runtime values, Cheap check, Agent, then Hybrid, then the other branch from Fixed steps to Costume, then Open loop.

The three shapes differ in one place: where the branch is resolved.

Chatbot. One model call per turn, possibly with retrieval and a fixed system prompt. The control flow is a request-response cycle you control. Conversation history is the only state. Cheap to build, cheap to test, and honest about being a language interface.

Workflow. You write the graph: classify, then route to handler A or B; extract, then validate, then write; call three tools in parallel and merge. Each node may be a model call, a function, or a human. The model makes local judgements inside nodes; it does not decide the topology. This is what most production "AI features" are, and should be.

Agent. The model emits the next action and the harness executes it. The topology is discovered. You constrain it with tools, budgets and stop conditions, and you evaluate it by trajectory rather than by output alone.

Real systems mix them. A support system might route with a classifier (workflow), answer from retrieval (chatbot), and hand a refund dispute to an agent with three tools and a human approval gate. That hybrid is usually the right answer: use the cheapest shape that covers the variability you actually have.

Three questions resolve the classification faster than a diagram:

  1. Can you write the steps as a decision tree that a new hire could follow? If yes, build the workflow.
  2. Does the correct next step depend on values you only learn by executing? Then part of it is agentic.
  3. Can you verify each step cheaply (tests, schema checks, read-back)? If not, autonomy compounds error instead of correcting it.

Trade-offs: where each shape wins

ShapeControl flowCost and latencyFails when
ChatbotFixed request-responseOne or two model calls, predictableThe task needs state changes, not answers
WorkflowAuthor-defined graphKnown node count, cacheableInputs fall outside the branches you drew
Workflow with one agent stepMostly fixed, one open nodeBounded by the agent node budgetThe agent node's failure is not contained
Agent with toolsModel-chosen sequenceLong-tailed, needs capsNo cheap verification of the end state
Multi-agentModel-chosen plus delegationMultiplies calls and coordination costSubtasks are sequential or share state
Trade-off mapInteractive

Good for

Fixed request-response

Cost

One or two model calls, predictable

Fails when

The task needs state changes, not answers

  1. The task needs state changes, not answers
  2. Inputs fall outside the branches you drew
  3. The agent node's failure is not contained
  4. No cheap verification of the end state
  5. Subtasks are sequential or share state

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Chatbot is good for Fixed request-response, costs One or two model calls, predictable, and fails when The task needs state changes, not answers. Workflow is good for Author-defined graph, costs Known node count, cacheable, and fails when Inputs fall outside the branches you drew. Workflow with one agent step is good for Mostly fixed, one open node, costs Bounded by the agent node budget, and fails when The agent node's failure is not contained. Agent with tools is good for Model-chosen sequence, costs Long-tailed, needs caps, and fails when No cheap verification of the end state. Multi-agent is good for Model-chosen plus delegation, costs Multiplies calls and coordination cost, and fails when Subtasks are sequential or share state

Failure modes

  • Workflow wearing an agent costume. A prompt says "first classify, then summarise, then write", and the model is trusted to follow it. Debug: check the trace for step ordering; if ordering matters, enforce it in code and keep the model inside nodes.
  • Agent used for a fixed procedure. Refund processing with four known steps runs as an open loop, so it sometimes skips the eligibility check. Debug: measure the variance in tool sequences across runs; near-zero variance means the sequence should be code.
  • Hybrid with no boundary. The agent can call the payment tool directly and the workflow also calls it, producing double charges. Debug: list every writer of each side-effecting resource; each should have exactly one owner.
  • Chatbot scope creep. A Q&A surface is asked to "just fix it", and the team bolts on tools without adding budgets or approval. Debug: compare the tool registry against the read-only set; every write tool needs a named owner and a gate.

In production

Workflows are cheap to operate because you can count nodes, cache deterministic steps and give each branch its own error budget. Agents invert that: you monitor distributions. Track steps per task, tool error rate, budget aborts and verified success. A workflow regression shows up as a specific node failing; an agent regression shows up as a shift in the distribution, which is why traces and a labelled task set are not optional.

There is also a staffing consequence. A workflow can be reviewed by reading the graph. An agent has to be reviewed by reading trajectories, so the team needs tooling to search and diff them before the first release, not after the first incident.

Common misconceptions

People think: If the model calls tools, it is an agent.

Actually: Tool calling is orthogonal. A workflow node can call a tool, and an agent can exist with a single tool. The distinguishing question is who chooses the order.

People think: Agents are the more advanced version, so build the agent and simplify later.

Actually: The reverse migration is the common one. Teams ship an agent, discover the task had four stable steps, and rewrite it as a workflow with a model call inside each step.

Interview angle

This is a favourite because it separates people who have operated a system from people who have read about one. Answer with the axis, not a definition: control-flow ownership. Then give the decision procedure (can a new hire follow the steps; does the next step depend on runtime results; can each step be verified) and one hybrid example. Interviewers often follow with "the task is open-ended but the actions are irreversible" — a good answer adds an approval gate rather than removing autonomy. See agents interview questions and when not to use an agent.

Sources

  • Anthropic, Building effective agents — the clearest published distinction between workflows and agents, with pattern-by-pattern guidance.
  • Huyen, Agents — control-flow taxonomy, failure modes and evaluation notes from an independent practitioner.
  • Sumers et al., Cognitive Architectures for Language Agents — separates decision procedure from action space, which maps onto the workflow-versus-agent split.
  • Yao et al., ReAct — the interleaved pattern that defines the agent end of the spectrum.