Skip to content
AI Engineer Guide.

concept · operations · intermediate · concept 27 of 38

Circuit breakers

0 of 38 complete

Why it matters

When a provider or downstream service degrades, continuing to call it consumes connection slots and threads, so the caller fails slowly instead of quickly and the dependency cannot recover. A breaker converts that into a bounded wait: after enough failures, calls return immediately, freeing capacity and giving the dependency room. The design questions are where the state lives, what counts as a failure, and how the probe is limited.

Mental model

It is a fuse in a circuit: it melts before the wiring does, and you replace it once. The analogy breaks because a software breaker resets itself, and the reset is the risky moment when every caller tries again at once.

How it works

Mechanism, step by stepInteractive
  1. 1Wrap one dependency, not a whole request, so a breaker on retrieval does not disable generation.
  2. 2Define what counts as a failure: connection errors, timeouts and specific status codes, but not user errors.
  3. 3Set the window and threshold from the dependency's normal error rate, so routine noise does not open the circuit.
  4. 4Fail fast while open with a typed error that the fallback ladder can handle.
  5. 5Allow a limited number of probe requests in half-open, and close only after consecutive successes.
  6. 6Export the state as a gauge and log transitions with the reason, so an incident timeline shows when the circuit changed.

Step 1 of 6

Wrap one dependency, not a whole request, so a breaker on retrieval does not disable generation.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Wrap one dependency, not a whole request, so a breaker on retrieval does not disable gene… 2. Define what counts as a failure: connection errors, timeouts and specific status codes, b… 3. Set the window and threshold from the dependency's normal error rate, so routine noise do… 4. Fail fast while open with a typed error that the fallback ladder can handle. 5. Allow a limited number of probe requests in half-open, and close only after consecutive s… 6. Export the state as a gauge and log transitions with the reason, so an incident timeline …

  1. Wrap one dependency, not a whole request, so a breaker on retrieval does not disable generation.
  2. Define what counts as a failure: connection errors, timeouts and specific status codes, but not user errors.
  3. Set the window and threshold from the dependency's normal error rate, so routine noise does not open the circuit.
  4. Fail fast while open with a typed error that the fallback ladder can handle.
  5. Allow a limited number of probe requests in half-open, and close only after consecutive successes.
  6. Export the state as a gauge and log transitions with the reason, so an incident timeline shows when the circuit changed.

Trade-offs

ChoiceGood forCostFails when
Per-process breakerSimple, no coordinationEach instance opens separatelyTraffic is uneven across instances
Shared breaker stateConsistent fleet behaviourA shared store on the failure pathThe store itself is unavailable
Low thresholdFast protectionTrips on normal noiseThe dependency has a naturally high error rate
Long open intervalGives the dependency time to recoverExtended degradation for usersThe dependency recovers quickly
Trade-off mapInteractive

Good for

Simple, no coordination

Cost

Each instance opens separately

Fails when

Traffic is uneven across instances

  1. Traffic is uneven across instances
  2. The store itself is unavailable
  3. The dependency has a naturally high error rate
  4. The dependency recovers quickly

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Per-process breaker is good for Simple, no coordination, costs Each instance opens separately, and fails when Traffic is uneven across instances. Shared breaker state is good for Consistent fleet behaviour, costs A shared store on the failure path, and fails when The store itself is unavailable. Low threshold is good for Fast protection, costs Trips on normal noise, and fails when The dependency has a naturally high error rate. Long open interval is good for Gives the dependency time to recover, costs Extended degradation for users, and fails when The dependency recovers quickly

Failure modes

  • Breaker without a timeout. Calls hang until the client timeout and the failure counter never increments. Debug: assert every wrapped call has a deadline shorter than the request budget.
  • Half-open flood. All waiting requests probe at once and knock the dependency over again. Debug: count probe requests per second after a transition to half-open.
  • Wrong failure classification. Validation errors from a healthy dependency open the circuit and disable a working path. Debug: chart breaker state against error codes and confirm the counts exclude client errors.

In production

Alert on time spent open, not just on transitions, and review the failure classification whenever a provider changes its error format. Rehearse recovery with a dependency stub that fails and then succeeds slowly.

Common misconceptions

People think: A circuit breaker improves availability by itself.

Actually: It shortens failure; availability comes from what the caller does while the circuit is open.

People think: Retries and breakers are alternatives.

Actually: Retries handle transient blips, breakers handle sustained failure; using only retries turns a degradation into a storm.

Interview angle

Walk the three states and the transition conditions, then the scope of the breaker and what the caller returns while it is open. A strong answer mentions timeouts and the limited probe. Weak answers describe a retry counter with no recovery policy.

Sources