Why it matters
Two providers can both accept a list of messages and return text, and still disagree on everything a real application depends on: whether tool calls arrive as complete objects or deltas, whether a refusal is a stop reason or an empty string, how cached tokens are counted, and which fields are optional. Applications that read provider responses directly accumulate those assumptions everywhere, and the cost appears when a fallback provider is used for the first time.
Mental model
It is a power adapter for different sockets: the appliance is the same, the plug is not, and one socket runs at a different voltage. The analogy breaks because voltage differs too, so some fields are not merely shaped differently but carry different meanings, and a plug that fits can still damage the device.
How it works
- 1Define the internal request and response types first, including tool calls, usage and finish reasons.
- 2Write one adapter per provider that translates in both directions and never leaks provider fields upward.
- 3Normalise streaming into a single event sequence: text delta, tool-call delta, usage, finish.
- 4Record real provider responses as fixtures and run the adapters against them in continuous integration.
- 5Map stop reasons into a closed internal enum, with an explicit unknown value that triggers logging.
- 6Keep token accounting per provider and use it for budgets, rather than estimating from characters.
Step 1 of 6
Define the internal request and response types first, including tool calls, usage and finish reasons.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Define the internal request and response types first, including tool calls, usage and fin… 2. Write one adapter per provider that translates in both directions and never leaks provide… 3. Normalise streaming into a single event sequence: text delta, tool-call delta, usage, fin… 4. Record real provider responses as fixtures and run the adapters against them in continuou… 5. Map stop reasons into a closed internal enum, with an explicit unknown value that trigger… 6. Keep token accounting per provider and use it for budgets, rather than estimating from ch…
- Define the internal request and response types first, including tool calls, usage and finish reasons.
- Write one adapter per provider that translates in both directions and never leaks provider fields upward.
- Normalise streaming into a single event sequence: text delta, tool-call delta, usage, finish.
- Record real provider responses as fixtures and run the adapters against them in continuous integration.
- Map stop reasons into a closed internal enum, with an explicit unknown value that triggers logging.
- Keep token accounting per provider and use it for budgets, rather than estimating from characters.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Thin adapter per provider | Clear mapping, easy to test | Duplicated logic across adapters | Providers add features frequently |
| Shared lowest-common-denominator schema | Simple application code | Loses provider-specific capabilities | A feature is the reason for choosing the provider |
| Pass-through with typed fields | Full access to new features | Leaks differences into the application | You need to switch providers |
| Generated clients from schemas | Up-to-date types | Generated code churn and weak semantics | The provider schema changes often |
Good for
Clear mapping, easy to test
Cost
Duplicated logic across adapters
Fails when
Providers add features frequently
- Providers add features frequently
- A feature is the reason for choosing the provider
- You need to switch providers
- The provider schema changes often
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Thin adapter per provider is good for Clear mapping, easy to test, costs Duplicated logic across adapters, and fails when Providers add features frequently. Shared lowest-common-denominator schema is good for Simple application code, costs Loses provider-specific capabilities, and fails when A feature is the reason for choosing the provider. Pass-through with typed fields is good for Full access to new features, costs Leaks differences into the application, and fails when You need to switch providers. Generated clients from schemas is good for Up-to-date types, costs Generated code churn and weak semantics, and fails when The provider schema changes often
Failure modes
- Tool-call id mismatch. Results are attached to the wrong call and the model loops or errors. Debug: assert every tool result carries an id present in the preceding call and log unmatched ids.
- Stop-reason mapping gap. A length limit is treated as a normal finish and truncated output is parsed as complete. Debug: enumerate observed finish reasons per provider and fail on any unmapped value.
- Usage field divergence. Budget calculations undercount because cached or reasoning tokens are reported in different fields. Debug: reconcile summed usage against the provider's invoice for a sample day.
In production
Run the conformance suite on every adapter change and on a schedule against a sandbox key. Alert on unmapped finish reasons, since that is the earliest signal that a provider changed its contract.
Common misconceptions
People think: A compatible endpoint means full compatibility.
Actually: It usually means the message and completion shape; tool calling, streaming details, usage fields and error semantics still differ.
People think: Token counts are comparable across providers.
Actually: Each tokeniser and accounting rule differs, so counts are per provider and must not be reused for another provider's budget.
Interview angle
Name the four structural differences, describe the internal contract and the fixture-based conformance test, and explain why usage must stay per provider. Assuming a shared API shape means shared behaviour is the trap.
Sources
- OpenAI, API reference — message, tool-call and usage shapes.
- Anthropic, Messages API — an independent message and tool-use format to map onto.
- JSON Schema — the schema vocabulary used to validate structured output.