Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 34 of 37

When not to use an agent

0 of 37 complete

Why it matters

Agents are the most expensive way to use a model: multiple calls, tool round trips, retries, and a long tail that resists capacity planning. They are also the hardest to evaluate, because the output can be right for the wrong reasons and the process is part of the product.

That cost is worth paying for a real class of problems. It is wasted on the much larger class where the steps are known. The decision deserves an explicit test rather than a preference, because the default in most teams has drifted toward "agent" for anything with a tool, and the migration back is expensive.

Mental model

Hiring a contractor versus following a recipe. If you have the recipe and the ingredients, cooking it yourself is faster and cheaper every time. You hire the contractor when the kitchen is unfamiliar, the dish is unusual, and you can taste the result before serving.

The analogy breaks on supervision. You can taste a dish in seconds; verifying an agent's work may require exactly the expertise you hired it to replace. When verification is as expensive as the work, autonomy has bought you nothing.

How it works

The facts that pick the shapeInteractive
startquestionoutcomepath takenruled out
VerdictChoose· recommended hereConditional· with the named costRefuse· decline the request

Each edge carries a fact read from the task, not a preference for the model; the branch that holds picks the shape.

Step 1 of 7

Write the facts down

Known steps, runtime branches, cheap verification, variance budget. The most specific shape that fits decides the shape, and each fact is a question a reviewer can challenge.

start

Which shape does the task need?

Read the facts about the task, not about the model: what it returns, whether the steps are known, whether a wrong result can be checked cheaply, whether the budget allows variance, and whether the path is genuinely open. The most specific shape that fits is the one to take.

Path walked

  1. Which shape does the task need?
Step 1/7 · Write the facts down

Figure. The shape is set by facts about the task, not by the model: what it returns, whether its steps are known, whether a wrong result can be checked cheaply, and whether the budget allows variance. Autonomy survives only where a cheap check, a capped budget and an open path all hold.

What to notice. The conditions overlap on purpose: an answer-only task also has known steps, so the more specific shape wins, exactly as the page's own chooser does. No latency, cost or confidence threshold appears because the page names none; its ten-a-day and ten-thousand illustration is not plotted. The figure does not claim the agent branch is never right — it claims autonomy is the last branch to justify.

Text version

A decision map from one question to six task shapes. The question is which shape the task needs, with four facts to write down: what the task returns, whether its steps are known, whether a wrong result can be checked cheaply, and whether the budget allows variance. The branch “answer only” leads to a single model call: classification, extraction or drafting, with no state and no recovery. “Lookup over a corpus” leads to retrieval plus generation, where retrieval quality is the ceiling. “Steps you can write” leads to a fixed workflow, with model calls inside code. “No cheap check” leads to a refusal: decline the loop and keep a person on the result, because verification costs as much as the work. “Variance is capped” leads to a bounded agent step inside deterministic code, conditional on a budget and a verifier. “Path unknown, checked” leads to an open agent, conditional on a queue, a budget and a cancellation path, for research and debugging. Each branch is selected by a fact about the task; the side panel walks one branch at a time.

Run these tests in order. The first one that fails is usually decisive.

  1. Can you write the steps? If a competent colleague could follow a numbered procedure, encode it. Model calls fit inside steps; the graph should not be a model decision.
  2. Does the next step depend on values discovered at runtime? If every branch is knowable from the input, you have a workflow. If the branch depends on what a file, API or test run reveals, that portion may be agentic.
  3. Can each step be verified? Tests, schema validation, read-back, a deterministic check. Without cheap verification, errors compound silently and the loop's adaptability becomes a liability.
  4. Is the action reversible? Irreversible actions need approval or a compensating action, which changes the design regardless of autonomy.
  5. Does the volume justify the variance? At ten tasks a day, a human in the loop is affordable. At ten thousand, tail latency and token spend dominate, and predictability matters more than coverage.
  6. Is the failure cheap? A wrong draft is cheap. A wrong refund, migration or email is not.

If the task is a single classification, extraction or rewrite over provided context, no loop is needed at all. If it is a lookup over a corpus, retrieval plus one generation call is the shape. If it is a fixed multi-step pipeline, a workflow. The agent is the last resort, not the first.

A rough arithmetic helps make the argument concrete. Suppose a workflow uses three model calls per task with modest output. An agent on the same task has a median of six steps and a p95 of thirty, and each step re-sends a transcript that grows. The capacity you must provision is set by the tail, while the value is delivered at the median. Teams routinely discover this after the first traffic spike, when the queue backs up and the budget alarm fires during the incident rather than before it.

def choose_shape(task) -> str:
    # Facts about the task, not about the model.
    if task.steps_are_known and task.branches_from_input_alone:
        return "workflow"
    if task.is_answer_only:                 # retrieval, classification, drafting
        return "single_call"
    if not task.can_be_verified_cheaply:
        return "human_in_the_loop"          # or do not automate yet
    if not task.budget_allows_variance:
        return "workflow_with_bounded_agent_step"
    return "agent"

The function is deliberately boring. Its value is that it forces the four facts to be written down, and each one is a question a reviewer can challenge. In practice the first branch resolves most requests, which is the point.

Trade-offs

OptionGood forCostFails when
Plain codeRules, validation, deterministic transformsBrittle on unstructured inputThe input is natural language or messy documents
Single model callClassification, extraction, draftingNo recovery, no stateThe answer needs data the prompt does not contain
Retrieval plus generationQuestions over a corpusRetrieval quality ceilingThe task requires an action, not an answer
Fixed workflowKnown procedures with model stepsBreaks outside its branchesInputs vary in structure, not just content
Bounded agent stepOne genuinely open sub-taskNeeds a budget and a verifierThe sub-task's output cannot be checked
Open agentResearch, debugging, explorationLong tail, trajectory evaluationYou need a guaranteed completion time
Trade-off mapInteractive

Good for

Rules, validation, deterministic transforms

Cost

Brittle on unstructured input

Fails when

The input is natural language or messy documents

  1. The input is natural language or messy documents
  2. The answer needs data the prompt does not contain
  3. The task requires an action, not an answer
  4. Inputs vary in structure, not just content
  5. The sub-task's output cannot be checked

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Plain code is good for Rules, validation, deterministic transforms, costs Brittle on unstructured input, and fails when The input is natural language or messy documents. Single model call is good for Classification, extraction, drafting, costs No recovery, no state, and fails when The answer needs data the prompt does not contain. Retrieval plus generation is good for Questions over a corpus, costs Retrieval quality ceiling, and fails when The task requires an action, not an answer. Fixed workflow is good for Known procedures with model steps, costs Breaks outside its branches, and fails when Inputs vary in structure, not just content. Bounded agent step is good for One genuinely open sub-task, costs Needs a budget and a verifier, and fails when The sub-task's output cannot be checked

Failure modes

  • Agent as a résumé item. The architecture was chosen for novelty, and the procedure it replaced was stable. Debug: write out the tool sequences from twenty traces; if fewer than three distinct sequences appear, replace the loop with code.
  • Unbounded cost per task. Nobody can state the p95 spend, and the budget cap is the provider's monthly limit. Debug: compute cost per resolved task from traces, then compare with the workflow equivalent; the difference is the price of autonomy.
  • Verification deferred to users. The agent ships without a checker, and quality is discovered through complaints. Debug: add an automated end-state check before adding capability; if none can be written, the task is not ready for autonomy.
  • Workflow in denial. A prompt lists the steps and the team calls it an agent, so no one builds the branch logic. Debug: check whether step order is enforced by code; if not, it is a suggestion.

In production

The operational difference is what you can promise. A workflow supports an SLO on latency and a fixed cost per request. An agent supports a distribution and a budget, and needs a queue, a concurrency limit and a cancellation path to avoid one slow run starving the rest.

Monitoring should make the cost of autonomy visible: steps per task, cost per resolved task, budget-abort rate, and the share of tasks that needed more than the median path. Review that last number quarterly. If the long tail is not delivering value, the task has converged to a procedure and the agent should be retired.

Common misconceptions

People think: If the task is hard, an agent is the right tool.

Actually: Difficulty and open-endedness are different axes. A hard but well-specified task is a workflow problem; an easy but open-ended task may need an agent.

People think: An agent can always be constrained later with guardrails.

Actually: Guardrails limit damage; they do not restore predictability. A loop with caps still has variable cost and a trajectory-shaped failure surface.

Interview angle

A frequent senior prompt: "Walk me through deciding between a workflow and an agent for a customer-support automation." Answer with the ordered tests, then commit to a shape and defend the cost. Interviewers notice whether you ask about verification before autonomy, and whether you can name what you would give up. See agents interview questions, agent vs workflow and cost modelling.

Sources