Why it matters
Every provider exposes the same broad idea, a prompt in and a completion out, with different field names, streaming events, error taxonomies and usage objects. Applications that call the SDK directly spread those details through the codebase. When a fallback provider is added or an SDK upgrade lands, the change touches every call site. An internal interface that normalises messages, tools, usage and errors is the difference between a one-file change and a migration.
Mental model
Provider APIs are national power sockets: the same electricity, different plugs. The comparison breaks because the voltage also changes, since token accounting, error retryability and streaming semantics are not equivalent across providers. An adapter is a transformer, not just a plug.
How it works
- 1Define an internal request type: messages, tools, sampling settings, stream flag and a metadata bag for provider-specific options.
- 2Normalise the response into content, tool calls, finish reason and usage, mapping each provider's fields into it.
- 3Map errors into a small taxonomy: retryable, rate limited, invalid request, content blocked, server error.
- 4Keep the SDK behind one adapter module and pin its version, so upgrades are a deliberate change.
- 5Record raw request and response payloads for a sample, because normalisation hides details you will need in an incident.
- 6Test the adapter against recorded fixtures rather than live calls, so CI is deterministic.
Step 1 of 6
Define an internal request type: messages, tools, sampling settings, stream flag and a metadata bag for provider-specific options.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Define an internal request type: messages, tools, sampling settings, stream flag and a me… 2. Normalise the response into content, tool calls, finish reason and usage, mapping each pr… 3. Map errors into a small taxonomy: retryable, rate limited, invalid request, content block… 4. Keep the SDK behind one adapter module and pin its version, so upgrades are a deliberate … 5. Record raw request and response payloads for a sample, because normalisation hides detail… 6. Test the adapter against recorded fixtures rather than live calls, so CI is deterministic.
- Define an internal request type: messages, tools, sampling settings, stream flag and a metadata bag for provider-specific options.
- Normalise the response into content, tool calls, finish reason and usage, mapping each provider's fields into it.
- Map errors into a small taxonomy: retryable, rate limited, invalid request, content blocked, server error.
- Keep the SDK behind one adapter module and pin its version, so upgrades are a deliberate change.
- Record raw request and response payloads for a sample, because normalisation hides details you will need in an incident.
- Test the adapter against recorded fixtures rather than live calls, so CI is deterministic.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Direct SDK calls | Least code, fastest start | Provider details spread everywhere | You add a second provider or upgrade an SDK |
| Thin internal adapter | Portable and testable | Some provider features need escape hatches | You depend on a unique capability |
| Gateway proxy | Central policy and logging | Another hop and a shared failure point | You need low latency or fine-grained control |
| Raw HTTP client | No SDK churn | You implement retries and streaming | The API surface is large |
Good for
Least code, fastest start
Cost
Provider details spread everywhere
Fails when
You add a second provider or upgrade an SDK
- You add a second provider or upgrade an SDK
- You depend on a unique capability
- You need low latency or fine-grained control
- The API surface is large
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Direct SDK calls is good for Least code, fastest start, costs Provider details spread everywhere, and fails when You add a second provider or upgrade an SDK. Thin internal adapter is good for Portable and testable, costs Some provider features need escape hatches, and fails when You depend on a unique capability. Gateway proxy is good for Central policy and logging, costs Another hop and a shared failure point, and fails when You need low latency or fine-grained control. Raw HTTP client is good for No SDK churn, costs You implement retries and streaming, and fails when The API surface is large
Failure modes
- Field drift after upgrade. A renamed field silently becomes undefined and a feature degrades. Debug: run adapter fixture tests and assert required fields are present.
- Error mapping too coarse. Rate limits are retried as server errors and amplify the throttle. Debug: log the provider error code and check the retry class assigned to it.
- Usage accounting mismatch. Cached or reasoning tokens are not counted, so cost dashboards understate spend. Debug: reconcile summed usage against the provider invoice for one day.
In production
Monitor error rate by normalised class, not by raw provider code, and keep a contract test suite that runs against each adapter. Log the SDK version with every request.
Common misconceptions
People think: All chat APIs are compatible enough to swap.
Actually: Message roles, tool schemas, streaming events and error semantics differ in ways that change behaviour.
People think: The SDK is the API.
Actually: It is a convenience wrapper; the wire format and its errors are what you must handle.
Interview angle
Describe the boundary you would build: normalised request, response, usage and error taxonomy, with fixtures for tests. Strong answers mention where provider-specific features escape the abstraction. Weak answers say they use the SDK and have no plan for a second provider or a breaking upgrade.
Sources
- OpenAI, Chat Completions API reference — request and response fields for a chat endpoint.
- Anthropic, Messages API reference — a different shape for the same concept.
- OpenAI, Error codes guide — which failures are retryable and which are not.