Why it matters
The schema is the only thing the model knows about a tool before it uses it. If the description says "search" and the tool actually queries a ticketing system with a required project key, the model will call it with a query string, fail, and try again. That is not a model failure; it is a specification failure.
The same object is the harness's enforcement point. Between the model's proposal and the side effect sits validation: does the name exist, do the arguments satisfy the schema, is this caller allowed to touch this resource. A tight schema turns a class of runtime incidents into a rejected call with a useful message.
Mental model
A form at a government office. The fields, their labels and the instructions in small print determine whether people fill it in correctly. The clerk at the window checks completeness and refuses incomplete forms.
The analogy breaks in two places. The model reads the form and the small print at once, so prose quality changes behaviour more than field ordering does. And unlike a clerk, the harness can also rewrite the form mid-conversation: if you change a tool's schema, every later turn sees the new version, including ones already in progress.
How it works
A tool schema is read twice: by the model as a prompt, by the harness as a validation contract. The two failures use different codes because they need different repairs.
Model
reads prose
Harness
validates
Tool
orders
{ "ok": false,
"code": "invalid_arguments",
"retryable": true,
"errors": [
{ "path": "$.statuss",
"error": "additional property not allowed" },
{ "path": "$.status",
"error": "must be one of open, shipped, cancelled" } ] }
Nothing executed. The message names the path
and the allowed values, which is what lets the
model repair instead of guessing.Step 1: reject field. Hover, tap or tab to any message to inspect its payload.
Figure. The harness is the enforcement point: invalid_arguments names the failing paths and executes nothing; unknown_tool is decided against the registry, not against the model's description. Both return as observations the model can repair from.
What to notice. The failure is a schema that is too permissive or a tool that no longer exists: additionalProperties false turns the invented statuss field into an error, and the registry rejects lookup_customer before any side effect. Description quality and schema drift between deploys are omitted. Emails, order ids and registry contents are illustrative.
Text version
Sequence across Model, Harness (validation and dispatch) and Tool (orders service). Step 1, reject field: an in-process Model to Harness row proposing search_orders with query ada@example.com, status shipping and an invented statuss field; Harness to Model an invalid_arguments error listing path $.statuss, additional property not allowed, and path $.status, must be one of open, shipped, cancelled, with retryable true and no execution. Step 2, repair: Model to Harness a corrected call with query, status shipped and limit 20. Step 3, execute: Harness to Tool search_orders with the validated arguments; Tool to Harness one match, ord_551 shipped. Step 4, observe: an in-process Harness to Model row carrying the result as data. Step 5, reject name: Model to Harness lookup_customer with email ada@example.com; Harness to Model an unknown_tool error with retryable false and registered tools search_orders and get_customer, never sent to the Tool.
- Name the action, not the system.
search_ordersbeatsorders_api. Names are tokens the model reasons over and greps in its own context. - Write the description for a competent stranger. State what it does, when to use it, when not to, and what the return contains. One or two sentences plus a note on limits.
- Constrain the parameters you can. Enums for closed sets, formats for dates, minimum and maximum for ranges, and
requiredonly for arguments that genuinely have no safe default. - Forbid the rest. Set
additionalProperties: falseso a hallucinated field is an error rather than silently ignored. - Define the result shape. Return one predictable object: a status, the data, and a short message. The model plans against the shape it saw in the description.
- Define the error shape. Include a machine-readable code, a human-readable message, and whether a retry could help. Errors are observations, not exceptions.
- Version explicitly. Renaming a field is a breaking change for every prompt and trace that references it.
SEARCH_SCHEMA = {
"name": "search_orders",
"description": (
"Search orders by customer email or order id. Returns at most 20 matches "
"with id, status, total_cents and created_at. Use get_order for full detail."
),
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Email address or order id prefix."},
"status": {"type": "string", "enum": ["open", "shipped", "cancelled"]},
"limit": {"type": "integer", "minimum": 1, "maximum": 20, "default": 10},
},
"required": ["query"],
"additionalProperties": False,
},
}
def dispatch(call, registry):
schema = registry.get(call.name)
if schema is None:
return {"ok": False, "code": "unknown_tool", "retryable": False}
errors = validate(call.args, schema["parameters"]) # JSON Schema validator
if errors:
return {"ok": False, "code": "invalid_arguments", "errors": errors, "retryable": True}
return registry.execute(call.name, call.args)The validation branch is the important one. Returning invalid_arguments with the specific failing path lets the model repair the call. Raising an exception and logging it does not: the model sees a generic failure and often repeats it.
Trade-offs
| Schema choice | Good for | Cost | Fails when |
|---|---|---|---|
| Many small, single-purpose tools | Precise calls, clear errors | More tokens and more selection errors | The registry grows past what fits comfortably |
| One broad tool with a mode field | Small registries, related operations | Model must set mode correctly | Modes have different argument shapes |
| Strict schema, additionalProperties false | Catching hallucinated fields | Breaks clients that add fields | You need forward-compatible extensions |
| Loose schema, validate in code | Rapid iteration | Errors surface late and vaguely | The side effect is irreversible |
| Structured error object | Self-repair by the model | More output tokens per failure | The error leaks internals |
Good for
Precise calls, clear errors
Cost
More tokens and more selection errors
Fails when
The registry grows past what fits comfortably
- The registry grows past what fits comfortably
- Modes have different argument shapes
- You need forward-compatible extensions
- The side effect is irreversible
- The error leaks internals
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Many small, single-purpose tools is good for Precise calls, clear errors, costs More tokens and more selection errors, and fails when The registry grows past what fits comfortably. One broad tool with a mode field is good for Small registries, related operations, costs Model must set mode correctly, and fails when Modes have different argument shapes. Strict schema, additionalProperties false is good for Catching hallucinated fields, costs Breaks clients that add fields, and fails when You need forward-compatible extensions. Loose schema, validate in code is good for Rapid iteration, costs Errors surface late and vaguely, and fails when The side effect is irreversible. Structured error object is good for Self-repair by the model, costs More output tokens per failure, and fails when The error leaks internals
Failure modes
- Two tools that sound the same.
get_customerandlookup_customerboth exist, and calls split unpredictably. Debug: cluster tool calls by name and inspect the ones with high error rates; if two names share a description, merge them. - Schema drift. The implementation adds a required field, the schema does not, and every call fails validation. Debug: generate schemas from the handler signatures and assert equality in CI, rather than maintaining two copies.
- Over-permissive
additionalProperties. A typo in a field name is ignored, and the tool runs with a default the model did not intend. Debug: log unvalidated keys at the boundary and fail closed in non-production. - Description that describes the implementation. "Calls the v2 endpoint with a bearer token" tells the model nothing about when to use it. Debug: give the description to someone who has not read the code and ask them to choose between two tools; if they hesitate, rewrite it.
In production
Treat the tool registry as a public API with a release process. Version it, diff it between deploys, and keep the old version callable while prompts and evaluations catch up. Log the exact schema hash with every trace so a regression can be attributed to a schema change rather than a model change.
Measure selection accuracy separately from execution accuracy: wrong tool, right tool with wrong arguments, right call with a failed effect. Those three need different fixes, and a single "tool error rate" hides which one is moving. Token cost of definitions belongs in the per-turn cost model, not in a footnote.
Common misconceptions
People think: The JSON Schema is the contract; the description is documentation.
Actually: For the model, the description is the contract. Most wrong calls come from ambiguous prose, not from violated types. Types catch malformed calls, not misconceived ones.
People think: More tools make the agent more capable.
Actually: Each tool adds selection ambiguity, tokens and a failure path. Capability usually improves when overlapping tools are removed.
Interview angle
A common prompt is "design the tool interface for an agent that manages orders". The strong answer starts with the action vocabulary (search, read, mutate), then the schema for each, then the error contract, then the permission model. Interviewers listen for whether you treat descriptions as prompts and errors as observations. A good follow-up answer explains why you would not expose a generic SQL tool to an agent. See agents interview questions, tool use and tool errors.
Sources
- Understanding JSON Schema — the constraint vocabulary used in tool definitions, including enums and additional properties.
- Model Context Protocol, Tools specification — a concrete, provider-neutral tool contract with input and output schemas.
- Anthropic, Writing effective tools for agents — practical guidance on naming, descriptions, token cost and consolidating overlapping tools.
- OpenAPI Initiative, OpenAPI Specification — the schema conventions most HTTP tool wrappers inherit.