Why it matters
Tools fail for different reasons: a timeout, a rate limit, a validation error, a missing permission, an absent record. Treating them alike produces either blind retries or silent success. The agent can only adapt to what appears in its observation, so the shape of the error decides whether the loop self-corrects or thrashes.
Mental model
An error message is a diagnostic note handed back to the operator, not a red light on a panel. The metaphor breaks because a red light says only that something is wrong, while the model needs the cause and the acceptable next action or it will guess.
How it works
Keyed transitions
- Model fix → Call: reissued
- Classify → Halt: auth ∨ internal
Event · step 1 of 10
Call · attempt 1
Entry: charge_card for order 8412, dispatched with an 8-second timeout. The call id is recorded before dispatch, so a failure can be matched to the attempt that produced it.
In state
- Call
Next: 503 classified
- Attempts
- 1
- Retries
- 0
- Errors
- 0
Figure. Nine states. Classify routes on retryable: transient errors go to a client retry, invalid input goes to the model with a field and its accepted values, and authorisation or internal errors stop at Halt. The red states are the page's failures: a storm on a deterministic 400, a lost detail that invites an identical call, and an empty result recorded as success.
What to notice. The 8-second timeout, the 200 ms backoff and the three-attempt cap are illustrative. Retry pacing, jitter, cap and deadline, is policy inside Retry rather than a separate state, and the figure draws one attempt at a time. A rate-limited 429 shares transient, and an unavailable tool shares Halt.
Text version
A state machine with nine states. Call is the start, a pill. Call goes to Classify, a hexagonal decision, on error, to Success, a terminal pill, on 2xx, and to Silent fallback, a red failure box, when an empty result is recorded as 2xx. Classify goes to Retry on transient, to Model fix on invalid input, to Halt, a red failure box, on a keyed failure edge whose guard is authorisation or internal, to Retry storm on a retried 400, and to Lost detail on an internal message. Retry has a repeat edge back to Call labelled attempt plus one. Model fix has a keyed repeat edge back to Call, guard reissued. The trace walks Call, Classify, Retry, Call, Classify, Model fix, Call, Success, then Retry storm, then Lost detail, then Silent fallback. Counters show attempts rising to 3 on the success path, retries to 1, and errors classified to 2.
- Define an error envelope: code, message, retryable flag, offending field and accepted values.
- Classify at the tool boundary: transient, invalid input, authorisation, not found, internal.
- Retry transient errors in the client with jittered backoff and a cap, not with model turns.
- Return correctable errors to the model with enough detail to change the call.
- Escalate or stop on authorisation and internal errors, which the model cannot fix.
- Log the error class with the tool and the run so repeated classes become visible.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Retry on any error | Simple policy | Multiplies load during an outage | The error is deterministic |
| Return every error to the model | Maximum chance of self-correction | Burns turns on unfixable faults | The failure is authorisation or infrastructure |
| Swallow and continue | The run completes | A wrong result is presented as success | The tool's output was required |
| Fail the whole run | No silent corruption | Loses partial progress | The failure is one optional lookup |
Good for
Simple policy
Cost
Multiplies load during an outage
Fails when
The error is deterministic
- The error is deterministic
- The failure is authorisation or infrastructure
- The tool's output was required
- The failure is one optional lookup
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Retry on any error is good for Simple policy, costs Multiplies load during an outage, and fails when The error is deterministic. Return every error to the model is good for Maximum chance of self-correction, costs Burns turns on unfixable faults, and fails when The failure is authorisation or infrastructure. Swallow and continue is good for The run completes, costs A wrong result is presented as success, and fails when The tool's output was required. Fail the whole run is good for No silent corruption, costs Loses partial progress, and fails when The failure is one optional lookup
Failure modes
- Silent fallback. A failed lookup returns an empty result and the agent answers as if nothing exists. Debug: assert the observation distinguishes empty from error.
- Retry storm. A deterministic 400 is retried until the budget ends. Debug: group errors by class and confirm non-retryable codes are never retried.
- Lost error detail. The model sees internal error and repeats the call unchanged. Debug: check that error observations include a code and a corrective hint.
In production
Monitor error rate by tool and class, retries per call and the share of runs ending on an error. Alert on a change in the class mix, which usually indicates an upstream contract change.
Common misconceptions
People think: Retries fix most tool failures.
Actually: Retries fix transient faults; they make deterministic errors more expensive.
People think: Showing the model a raw stack trace helps it recover.
Actually: It usually produces a paraphrase and another identical call; a structured cause and fix does better.
Interview angle
Strong answers classify errors by who can fix them, put client retries where they belong, and describe the envelope the model sees. Weak answers say errors are logged and the agent tries again.
Sources
- AWS Architecture Blog, Exponential Backoff and Jitter — retry pacing for transient failures.
- RFC 9110, HTTP Semantics — status code classes and their retry semantics.
- Anthropic, Tool use with Claude — returning tool results and errors into the conversation.