Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 26 of 28

Provider APIs

0 of 28 complete

Why it matters

Every provider exposes the same broad idea, a prompt in and a completion out, with different field names, streaming events, error taxonomies and usage objects. Applications that call the SDK directly spread those details through the codebase. When a fallback provider is added or an SDK upgrade lands, the change touches every call site. An internal interface that normalises messages, tools, usage and errors is the difference between a one-file change and a migration.

Mental model

Provider APIs are national power sockets: the same electricity, different plugs. The comparison breaks because the voltage also changes, since token accounting, error retryability and streaming semantics are not equivalent across providers. An adapter is a transformer, not just a plug.

How it works

Mechanism, step by stepInteractive
  1. 1Define an internal request type: messages, tools, sampling settings, stream flag and a metadata bag for provider-specific options.
  2. 2Normalise the response into content, tool calls, finish reason and usage, mapping each provider's fields into it.
  3. 3Map errors into a small taxonomy: retryable, rate limited, invalid request, content blocked, server error.
  4. 4Keep the SDK behind one adapter module and pin its version, so upgrades are a deliberate change.
  5. 5Record raw request and response payloads for a sample, because normalisation hides details you will need in an incident.
  6. 6Test the adapter against recorded fixtures rather than live calls, so CI is deterministic.

Step 1 of 6

Define an internal request type: messages, tools, sampling settings, stream flag and a metadata bag for provider-specific options.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Define an internal request type: messages, tools, sampling settings, stream flag and a me… 2. Normalise the response into content, tool calls, finish reason and usage, mapping each pr… 3. Map errors into a small taxonomy: retryable, rate limited, invalid request, content block… 4. Keep the SDK behind one adapter module and pin its version, so upgrades are a deliberate … 5. Record raw request and response payloads for a sample, because normalisation hides detail… 6. Test the adapter against recorded fixtures rather than live calls, so CI is deterministic.

  1. Define an internal request type: messages, tools, sampling settings, stream flag and a metadata bag for provider-specific options.
  2. Normalise the response into content, tool calls, finish reason and usage, mapping each provider's fields into it.
  3. Map errors into a small taxonomy: retryable, rate limited, invalid request, content blocked, server error.
  4. Keep the SDK behind one adapter module and pin its version, so upgrades are a deliberate change.
  5. Record raw request and response payloads for a sample, because normalisation hides details you will need in an incident.
  6. Test the adapter against recorded fixtures rather than live calls, so CI is deterministic.

Trade-offs

ChoiceGood forCostFails when
Direct SDK callsLeast code, fastest startProvider details spread everywhereYou add a second provider or upgrade an SDK
Thin internal adapterPortable and testableSome provider features need escape hatchesYou depend on a unique capability
Gateway proxyCentral policy and loggingAnother hop and a shared failure pointYou need low latency or fine-grained control
Raw HTTP clientNo SDK churnYou implement retries and streamingThe API surface is large
Trade-off mapInteractive

Good for

Least code, fastest start

Cost

Provider details spread everywhere

Fails when

You add a second provider or upgrade an SDK

  1. You add a second provider or upgrade an SDK
  2. You depend on a unique capability
  3. You need low latency or fine-grained control
  4. The API surface is large

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Direct SDK calls is good for Least code, fastest start, costs Provider details spread everywhere, and fails when You add a second provider or upgrade an SDK. Thin internal adapter is good for Portable and testable, costs Some provider features need escape hatches, and fails when You depend on a unique capability. Gateway proxy is good for Central policy and logging, costs Another hop and a shared failure point, and fails when You need low latency or fine-grained control. Raw HTTP client is good for No SDK churn, costs You implement retries and streaming, and fails when The API surface is large

Failure modes

  • Field drift after upgrade. A renamed field silently becomes undefined and a feature degrades. Debug: run adapter fixture tests and assert required fields are present.
  • Error mapping too coarse. Rate limits are retried as server errors and amplify the throttle. Debug: log the provider error code and check the retry class assigned to it.
  • Usage accounting mismatch. Cached or reasoning tokens are not counted, so cost dashboards understate spend. Debug: reconcile summed usage against the provider invoice for one day.

In production

Monitor error rate by normalised class, not by raw provider code, and keep a contract test suite that runs against each adapter. Log the SDK version with every request.

Common misconceptions

People think: All chat APIs are compatible enough to swap.

Actually: Message roles, tool schemas, streaming events and error semantics differ in ways that change behaviour.

People think: The SDK is the API.

Actually: It is a convenience wrapper; the wire format and its errors are what you must handle.

Interview angle

Describe the boundary you would build: normalised request, response, usage and error taxonomy, with fixtures for tests. Strong answers mention where provider-specific features escape the abstraction. Weak answers say they use the SDK and have no plan for a second provider or a breaking upgrade.

Sources