Skip to content
AI Engineer Guide.

concept · operations · advanced · concept 22 of 38

OTel GenAI

0 of 38 complete

Why it matters

Every observability vendor invented its own attribute names for the same model call, so a trace from one SDK could not be compared with a trace from a gateway. The GenAI conventions give a shared vocabulary: which operation happened, which provider and model were used, how many tokens were consumed and why generation stopped. Without that vocabulary, cross-service analysis becomes a mapping project that repeats on every backend migration.

Mental model

It is a standard shipping label: the carrier, the service and the weight go in fixed fields so any depot can read them. The analogy breaks because the label format is still being revised, so a parcel can arrive with fields an older depot does not recognise.

How it works

Mechanism, step by stepInteractive
  1. 1Instrument at one boundary, usually the gateway or a shared client, so every model call emits the same span shape.
  2. 2Name spans from the operation, and set the provider, model and request attributes the convention defines.
  3. 3Record token usage from the response, not from an estimate, and keep the estimate only as a fallback attribute.
  4. 4Keep prompt and completion content out of attributes by default; use an event or an external reference under an explicit capture policy.
  5. 5Pin the convention version in code and in exporter configuration, and note it in the trace schema documentation.
  6. 6Test the mapping with a recorded request so an SDK upgrade that renames an attribute fails a test rather than a dashboard.

Step 1 of 6

Instrument at one boundary, usually the gateway or a shared client, so every model call emits the same span shape.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Instrument at one boundary, usually the gateway or a shared client, so every model call e… 2. Name spans from the operation, and set the provider, model and request attributes the con… 3. Record token usage from the response, not from an estimate, and keep the estimate only as… 4. Keep prompt and completion content out of attributes by default; use an event or an exter… 5. Pin the convention version in code and in exporter configuration, and note it in the trac… 6. Test the mapping with a recorded request so an SDK upgrade that renames an attribute fail…

  1. Instrument at one boundary, usually the gateway or a shared client, so every model call emits the same span shape.
  2. Name spans from the operation, and set the provider, model and request attributes the convention defines.
  3. Record token usage from the response, not from an estimate, and keep the estimate only as a fallback attribute.
  4. Keep prompt and completion content out of attributes by default; use an event or an external reference under an explicit capture policy.
  5. Pin the convention version in code and in exporter configuration, and note it in the trace schema documentation.
  6. Test the mapping with a recorded request so an SDK upgrade that renames an attribute fails a test rather than a dashboard.

Trade-offs

ChoiceGood forCostFails when
Conventions at the gatewayUniform spans across providersApplication context must be added separatelyThe failure is inside application logic
Conventions in each SDKRich local contextVersion skew between servicesTeams upgrade at different times
Content in attributesAnswers are self-containedSpan size and privacy riskPrompts contain personal data
Custom attributes alongsideFits existing dashboardsTwo vocabularies to maintainNobody documents which is authoritative
Trade-off mapInteractive

Good for

Uniform spans across providers

Cost

Application context must be added separately

Fails when

The failure is inside application logic

  1. The failure is inside application logic
  2. Teams upgrade at different times
  3. Prompts contain personal data
  4. Nobody documents which is authoritative

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Conventions at the gateway is good for Uniform spans across providers, costs Application context must be added separately, and fails when The failure is inside application logic. Conventions in each SDK is good for Rich local context, costs Version skew between services, and fails when Teams upgrade at different times. Content in attributes is good for Answers are self-contained, costs Span size and privacy risk, and fails when Prompts contain personal data. Custom attributes alongside is good for Fits existing dashboards, costs Two vocabularies to maintain, and fails when Nobody documents which is authoritative

Failure modes

  • Attribute drift after upgrade. An SDK renames a field and token dashboards go blank. Debug: assert required attribute keys in an integration test against a recorded span.
  • High-cardinality metrics. User or session ids are put in metric labels and the time series count explodes. Debug: list distinct label values per metric and move identifiers to spans.
  • Version mismatch across services. Half the fleet emits the old convention and joins fail. Debug: aggregate spans by convention version and alert on mixed versions during a rollout.

In production

Treat the convention version as part of your schema: record it, test it and roll it out like a model change. Watch exporter drop rates, since oversized attribute sets fail silently at the collector.

Common misconceptions

People think: Adopting OpenTelemetry means traces are automatically comparable.

Actually: Portability comes from agreeing on attribute names and versions, which is exactly what the GenAI conventions add.

People think: The conventions require capturing prompts.

Actually: Content capture is optional and governed by policy; the required attributes describe the operation, model and usage.

Interview angle

Name the operation, provider, model and usage attributes and explain why a shared vocabulary matters for cross-service joins. Mention version pinning and content policy. Saying you use OpenTelemetry and stopping at the exporter misses the convention entirely.

Sources