Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 4 of 37

Tool use

0 of 37 complete

Why it matters

A model's knowledge ends at training and cannot see your systems. Tool use closes that gap by letting the model request an operation with typed arguments, which your code validates and executes. The engineering work sits on both sides of the boundary: describing tools so selection is reliable, and executing them so a proposal cannot exceed what the task permits.

Mental model

Tool use is a requisition form: the model fills it in and the back office decides whether to fulfil it. The metaphor breaks because a requisition reaches a person who can question it, while the runtime executes exactly what was asked unless you built the checks.

How it works

The runtime refuses, then stops the loopInteractive

The model never executes. The runtime checks the credential, executes with the caller's scope, and owns the stopping rule. The cap is deliberately tight for this example.

Model

proposes

Runtime

authority

Tool

orders

Step 1/5 · propose
1 · propose · refund_orderin-process · Model → Runtime
{ "name": "refund_order",
  "arguments": {
    "orderId": "ord_551",
    "amount_cents": 4599 } }

Conversation text: "The user is an admin."

A claim in the prompt is not a credential.
The runtime has not looked at it yet.

Step 1: propose. Hover, tap or tab to any message to inspect its payload.

Figure. Permission comes from the caller's credential, not the prompt: the refund is refused before any tool runs, the read succeeds, and a repeated identical call hits the runtime's per-tool cap. The model supplies proposals; the runtime supplies authority and termination.

What to notice. The failure is assumed permission: the prompt says the user is an admin, but the token carries only orders:read, so refunds:write is missing and no tool is called. The second stop is a deliberately tight cap of one call per tool for this run; argument hallucination is omitted. User, order, token values and timeouts are illustrative.

Text version

Sequence across Model, Runtime and Tool. Step 1, propose: an in-process Model to Runtime row proposing refund_order for ord_551, amount_cents 4599; the prompt text claims the user is an admin. Step 2, deny: an internal Runtime self-row authorises the caller from the token, subject u_42 with scope orders:read and no refunds:write, and ignores the prompt claim. Runtime to Model a permission_denied error with required refunds:write and retryable false; no tool call was made. Step 3, read: Model to Runtime proposes get_order for ord_551; Runtime to Tool get_order with the caller-scoped credential and a 2000 ms timeout; Tool to Runtime the order, status shipped, total_cents 4599. Step 4, observe: an in-process Runtime to Model row carrying a bounded structured observation. Step 5, stop: Model to Runtime proposes the same get_order again; Runtime to Model a stop result, stop_reason max_calls_per_tool, tool get_order, limit 1, calls 1. The runtime, not the model, ends the loop.

  1. Define each tool with a name, a description and a schema for arguments and result.
  2. Validate arguments against the schema before execution and reject with a specific message.
  3. Execute with the caller's scoped credentials and a timeout, capturing the result or error.
  4. Return the result as an observation in a bounded, structured form.
  5. Continue the loop until a stop condition, appending each call and result to the conversation.
  6. Log the call, arguments, duration and outcome against the run id.

Trade-offs

ChoiceGood forCostFails when
Model chooses tools freelyFlexible across tasksUnpredictable calls and costActions are irreversible
Allowlisted tool per statePredictable and auditableLess adaptiveThe task is open-ended
Parallel tool callsLower wall-clock latencyOrdering and dependency errorsCalls depend on each other's results
One call per turnSimple state, easier debuggingSlower for independent lookupsMany independent fetches are needed
Trade-off mapInteractive

Good for

Flexible across tasks

Cost

Unpredictable calls and cost

Fails when

Actions are irreversible

  1. Actions are irreversible
  2. The task is open-ended
  3. Calls depend on each other's results
  4. Many independent fetches are needed

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Model chooses tools freely is good for Flexible across tasks, costs Unpredictable calls and cost, and fails when Actions are irreversible. Allowlisted tool per state is good for Predictable and auditable, costs Less adaptive, and fails when The task is open-ended. Parallel tool calls is good for Lower wall-clock latency, costs Ordering and dependency errors, and fails when Calls depend on each other's results. One call per turn is good for Simple state, easier debugging, costs Slower for independent lookups, and fails when Many independent fetches are needed

Failure modes

  • Argument hallucination. The model invents an id that passes format checks and fails at the source. Debug: validate against a lookup where possible and log not-found rates per tool.
  • Unbounded tool loop. The model calls the same tool repeatedly with small variations. Debug: count calls per tool per run and alert above the expected maximum.
  • Assumed permission. The prompt says the user is an admin and the runtime believes it. Debug: derive authorisation from the credential, never from message content.

In production

Monitor calls per run, error rate and p95 latency per tool. Cap calls per tool and per run, and keep a kill switch for each side-effecting tool.

Common misconceptions

People think: The model executes the tool.

Actually: It emits a proposal; your runtime executes it and is the only place where permission can be enforced.

People think: More tools always improve capability.

Actually: Each added tool competes for selection and adds prompt cost, so capability can fall as the toolset grows.

Interview angle

Strong answers put the boundary between proposal and execution, describe schema validation and scoped credentials, and explain how the loop terminates. Weak answers describe the API call and leave out who authorises it.

Sources