Why it matters
A model's knowledge ends at training and cannot see your systems. Tool use closes that gap by letting the model request an operation with typed arguments, which your code validates and executes. The engineering work sits on both sides of the boundary: describing tools so selection is reliable, and executing them so a proposal cannot exceed what the task permits.
Mental model
Tool use is a requisition form: the model fills it in and the back office decides whether to fulfil it. The metaphor breaks because a requisition reaches a person who can question it, while the runtime executes exactly what was asked unless you built the checks.
How it works
The model never executes. The runtime checks the credential, executes with the caller's scope, and owns the stopping rule. The cap is deliberately tight for this example.
Model
proposes
Runtime
authority
Tool
orders
{ "name": "refund_order",
"arguments": {
"orderId": "ord_551",
"amount_cents": 4599 } }
Conversation text: "The user is an admin."
A claim in the prompt is not a credential.
The runtime has not looked at it yet.Step 1: propose. Hover, tap or tab to any message to inspect its payload.
Figure. Permission comes from the caller's credential, not the prompt: the refund is refused before any tool runs, the read succeeds, and a repeated identical call hits the runtime's per-tool cap. The model supplies proposals; the runtime supplies authority and termination.
What to notice. The failure is assumed permission: the prompt says the user is an admin, but the token carries only orders:read, so refunds:write is missing and no tool is called. The second stop is a deliberately tight cap of one call per tool for this run; argument hallucination is omitted. User, order, token values and timeouts are illustrative.
Text version
Sequence across Model, Runtime and Tool. Step 1, propose: an in-process Model to Runtime row proposing refund_order for ord_551, amount_cents 4599; the prompt text claims the user is an admin. Step 2, deny: an internal Runtime self-row authorises the caller from the token, subject u_42 with scope orders:read and no refunds:write, and ignores the prompt claim. Runtime to Model a permission_denied error with required refunds:write and retryable false; no tool call was made. Step 3, read: Model to Runtime proposes get_order for ord_551; Runtime to Tool get_order with the caller-scoped credential and a 2000 ms timeout; Tool to Runtime the order, status shipped, total_cents 4599. Step 4, observe: an in-process Runtime to Model row carrying a bounded structured observation. Step 5, stop: Model to Runtime proposes the same get_order again; Runtime to Model a stop result, stop_reason max_calls_per_tool, tool get_order, limit 1, calls 1. The runtime, not the model, ends the loop.
- Define each tool with a name, a description and a schema for arguments and result.
- Validate arguments against the schema before execution and reject with a specific message.
- Execute with the caller's scoped credentials and a timeout, capturing the result or error.
- Return the result as an observation in a bounded, structured form.
- Continue the loop until a stop condition, appending each call and result to the conversation.
- Log the call, arguments, duration and outcome against the run id.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Model chooses tools freely | Flexible across tasks | Unpredictable calls and cost | Actions are irreversible |
| Allowlisted tool per state | Predictable and auditable | Less adaptive | The task is open-ended |
| Parallel tool calls | Lower wall-clock latency | Ordering and dependency errors | Calls depend on each other's results |
| One call per turn | Simple state, easier debugging | Slower for independent lookups | Many independent fetches are needed |
Good for
Flexible across tasks
Cost
Unpredictable calls and cost
Fails when
Actions are irreversible
- Actions are irreversible
- The task is open-ended
- Calls depend on each other's results
- Many independent fetches are needed
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Model chooses tools freely is good for Flexible across tasks, costs Unpredictable calls and cost, and fails when Actions are irreversible. Allowlisted tool per state is good for Predictable and auditable, costs Less adaptive, and fails when The task is open-ended. Parallel tool calls is good for Lower wall-clock latency, costs Ordering and dependency errors, and fails when Calls depend on each other's results. One call per turn is good for Simple state, easier debugging, costs Slower for independent lookups, and fails when Many independent fetches are needed
Failure modes
- Argument hallucination. The model invents an id that passes format checks and fails at the source. Debug: validate against a lookup where possible and log not-found rates per tool.
- Unbounded tool loop. The model calls the same tool repeatedly with small variations. Debug: count calls per tool per run and alert above the expected maximum.
- Assumed permission. The prompt says the user is an admin and the runtime believes it. Debug: derive authorisation from the credential, never from message content.
In production
Monitor calls per run, error rate and p95 latency per tool. Cap calls per tool and per run, and keep a kill switch for each side-effecting tool.
Common misconceptions
People think: The model executes the tool.
Actually: It emits a proposal; your runtime executes it and is the only place where permission can be enforced.
People think: More tools always improve capability.
Actually: Each added tool competes for selection and adds prompt cost, so capability can fall as the toolset grows.
Interview angle
Strong answers put the boundary between proposal and execution, describe schema validation and scoped credentials, and explain how the loop terminates. Weak answers describe the API call and leave out who authorises it.
Sources
- Schick et al., Toolformer: Language Models Can Teach Themselves to Use Tools — models emitting API calls and consuming their results.
- Anthropic, Tool use with Claude — the request, call and result cycle in a production API.
- Model Context Protocol specification — a standard envelope for exposing tools to a model.