Skip to content
AI Engineer Guide.

concept · operations · intermediate · concept 30 of 38

Provider compatibility

0 of 38 complete

Why it matters

Two providers can both accept a list of messages and return text, and still disagree on everything a real application depends on: whether tool calls arrive as complete objects or deltas, whether a refusal is a stop reason or an empty string, how cached tokens are counted, and which fields are optional. Applications that read provider responses directly accumulate those assumptions everywhere, and the cost appears when a fallback provider is used for the first time.

Mental model

It is a power adapter for different sockets: the appliance is the same, the plug is not, and one socket runs at a different voltage. The analogy breaks because voltage differs too, so some fields are not merely shaped differently but carry different meanings, and a plug that fits can still damage the device.

How it works

Mechanism, step by stepInteractive
  1. 1Define the internal request and response types first, including tool calls, usage and finish reasons.
  2. 2Write one adapter per provider that translates in both directions and never leaks provider fields upward.
  3. 3Normalise streaming into a single event sequence: text delta, tool-call delta, usage, finish.
  4. 4Record real provider responses as fixtures and run the adapters against them in continuous integration.
  5. 5Map stop reasons into a closed internal enum, with an explicit unknown value that triggers logging.
  6. 6Keep token accounting per provider and use it for budgets, rather than estimating from characters.

Step 1 of 6

Define the internal request and response types first, including tool calls, usage and finish reasons.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Define the internal request and response types first, including tool calls, usage and fin… 2. Write one adapter per provider that translates in both directions and never leaks provide… 3. Normalise streaming into a single event sequence: text delta, tool-call delta, usage, fin… 4. Record real provider responses as fixtures and run the adapters against them in continuou… 5. Map stop reasons into a closed internal enum, with an explicit unknown value that trigger… 6. Keep token accounting per provider and use it for budgets, rather than estimating from ch…

  1. Define the internal request and response types first, including tool calls, usage and finish reasons.
  2. Write one adapter per provider that translates in both directions and never leaks provider fields upward.
  3. Normalise streaming into a single event sequence: text delta, tool-call delta, usage, finish.
  4. Record real provider responses as fixtures and run the adapters against them in continuous integration.
  5. Map stop reasons into a closed internal enum, with an explicit unknown value that triggers logging.
  6. Keep token accounting per provider and use it for budgets, rather than estimating from characters.

Trade-offs

ChoiceGood forCostFails when
Thin adapter per providerClear mapping, easy to testDuplicated logic across adaptersProviders add features frequently
Shared lowest-common-denominator schemaSimple application codeLoses provider-specific capabilitiesA feature is the reason for choosing the provider
Pass-through with typed fieldsFull access to new featuresLeaks differences into the applicationYou need to switch providers
Generated clients from schemasUp-to-date typesGenerated code churn and weak semanticsThe provider schema changes often
Trade-off mapInteractive

Good for

Clear mapping, easy to test

Cost

Duplicated logic across adapters

Fails when

Providers add features frequently

  1. Providers add features frequently
  2. A feature is the reason for choosing the provider
  3. You need to switch providers
  4. The provider schema changes often

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Thin adapter per provider is good for Clear mapping, easy to test, costs Duplicated logic across adapters, and fails when Providers add features frequently. Shared lowest-common-denominator schema is good for Simple application code, costs Loses provider-specific capabilities, and fails when A feature is the reason for choosing the provider. Pass-through with typed fields is good for Full access to new features, costs Leaks differences into the application, and fails when You need to switch providers. Generated clients from schemas is good for Up-to-date types, costs Generated code churn and weak semantics, and fails when The provider schema changes often

Failure modes

  • Tool-call id mismatch. Results are attached to the wrong call and the model loops or errors. Debug: assert every tool result carries an id present in the preceding call and log unmatched ids.
  • Stop-reason mapping gap. A length limit is treated as a normal finish and truncated output is parsed as complete. Debug: enumerate observed finish reasons per provider and fail on any unmapped value.
  • Usage field divergence. Budget calculations undercount because cached or reasoning tokens are reported in different fields. Debug: reconcile summed usage against the provider's invoice for a sample day.

In production

Run the conformance suite on every adapter change and on a schedule against a sandbox key. Alert on unmapped finish reasons, since that is the earliest signal that a provider changed its contract.

Common misconceptions

People think: A compatible endpoint means full compatibility.

Actually: It usually means the message and completion shape; tool calling, streaming details, usage fields and error semantics still differ.

People think: Token counts are comparable across providers.

Actually: Each tokeniser and accounting rule differs, so counts are per provider and must not be reused for another provider's budget.

Interview angle

Name the four structural differences, describe the internal contract and the fixture-based conformance test, and explain why usage must stay per provider. Assuming a shared API shape means shared behaviour is the trap.

Sources