Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 19 of 28

Tool calling

0 of 28 complete

Why it matters

A model cannot read a database or send an email, and prompting it to produce prose that your code then parses is fragile. Tool calling makes the interface explicit: the model chooses from declared functions and fills a schema, the application executes and returns data. The engineering problem shifts to the boundary, where model output becomes an action, and that boundary needs validation, permissions and a stopping rule.

Mental model

Tool calling is a requisition form: the model fills in which form and which fields, and a system decides whether to fulfil it. The form metaphor breaks because the model can invent a form that does not exist or fill a field with a value that passes the schema and is still wrong.

How it works

Two calls, one partial failureInteractive

Tool calling is a loop with two turns: tools in, calls out, results back. Every call id needs exactly one result, including a failure.

Model

provider

App

validates

Tool

functions

Step 1/4 · request
2 · two calls · parallelreturn · Model → App
{ "choices": [{ "message": {
  "role": "assistant",
  "tool_calls": [
    { "id": "call_1", "type": "function",
      "function": { "name": "get_order",
        "arguments":
          "{\"orderId\":\"ord_551\"}" } },
    { "id": "call_2", "type": "function",
      "function": { "name": "get_customer",
        "arguments":
          "{\"email\":\"ada@example.com\"}" } } ] },
  "finish_reason": "tool_calls" }] }

Two independent calls in one message. The model
has not executed anything.

Step 1: request. Hover, tap or tab to any message to inspect its payload.

Figure. The model proposes two independent calls in one turn; the application validates and executes both, and answers both ids in the next turn. One times out, and the timeout is a tool result, not an exception, so the loop continues with a known gap.

What to notice. The failure is a per-tool timeout: a parallel call that does not answer must still be answered, or the next model turn has an unmatched call id. A schema-valid but wrong argument and a runaway loop are omitted; the iteration cap is not reached. Order ids, amounts and timings are illustrative.

Text version

Sequence across Model (provider), App (validation and execution) and Tool (functions). Step 1, request: App to Model a chat request with the user message 'Where is order 551?' and two function tools, get_order and get_customer. Step 2, parallel: Model to App two tool_call entries in one assistant message, call_1 get_order with arguments '{"orderId":"ord_551"}' and call_2 get_customer with arguments '{"email":"ada@example.com"}', finish_reason tool_calls. Step 3, execute: App to Tool get_order and, while it is running, App to Tool get_customer with a 2000 ms timeout. Step 4, results: Tool to App the order status shipped and total_cents 4599, then an error after 2000 ms for get_customer. The app records a tool error rather than raising an exception. Step 5, result turn: App to Model the assistant tool_calls message plus two role tool messages, one with the order JSON and one with ok false, code timeout and retryable true, each carrying its call id. Step 6, answer: Model to App a final answer saying order 551 shipped and the customer lookup timed out, so tier cannot be confirmed.

  1. Define each tool with a narrow purpose, a description written for the model, and a schema that rejects what you cannot handle.
  2. Pass the tool definitions with the request and let the model return a call or a normal message.
  3. Validate the call against the schema and against policy before executing anything.
  4. Execute with a timeout and return either a result or a structured error in the tool result turn.
  5. Append the result and call the model again, with a hard cap on iterations and total tool calls.
  6. Log the chosen tool, arguments and outcome per step so selection errors are separable from execution errors.

Trade-offs

ChoiceGood forCostFails when
Few broad toolsSmall prompt, flexibleAmbiguous selection and argument errorsSeveral actions share similar inputs
Many narrow toolsPredictable selectionLarge schema in every request, more confusionThe catalogue grows past a few dozen
Model chooses freelyHandles novel requestsUnpredictable cost and side effectsActions are irreversible
Application routes to one toolDeterministic and cheapNo flexibilityThe request type is genuinely unknown
Trade-off mapInteractive

Good for

Small prompt, flexible

Cost

Ambiguous selection and argument errors

Fails when

Several actions share similar inputs

  1. Several actions share similar inputs
  2. The catalogue grows past a few dozen
  3. Actions are irreversible
  4. The request type is genuinely unknown

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Few broad tools is good for Small prompt, flexible, costs Ambiguous selection and argument errors, and fails when Several actions share similar inputs. Many narrow tools is good for Predictable selection, costs Large schema in every request, more confusion, and fails when The catalogue grows past a few dozen. Model chooses freely is good for Handles novel requests, costs Unpredictable cost and side effects, and fails when Actions are irreversible. Application routes to one tool is good for Deterministic and cheap, costs No flexibility, and fails when The request type is genuinely unknown

Failure modes

  • Schema-valid but wrong call. The model passes a plausible customer id that belongs to someone else. Debug: log arguments and verify identity against the authenticated session, not the model's claim.
  • Tool loop. The model calls the same tool repeatedly because the result is unhelpful. Debug: count calls per turn and inspect the result text the model received.
  • Description-driven misselection. Two tools with overlapping descriptions get confused. Debug: review the description text and test a labelled set of requests per tool.

In production

Cap tool calls per request and per session, set per-tool timeouts, and monitor selection accuracy separately from execution errors. Treat every tool result as untrusted input when it re-enters the prompt.

Common misconceptions

People think: The model runs the function.

Actually: It only emits a structured proposal; your code decides whether and how to execute it.

People think: A valid schema means the call is safe.

Actually: Schema validity checks shape, not intent, authorisation or side effects.

Interview angle

Describe the loop and then concentrate on the trust boundary: validation, authorisation, timeouts and iteration caps. Strong answers separate selection accuracy from execution reliability. Weak answers say the model calls APIs and skip the fact that every call is a proposal until your code approves it.

Sources