Why it matters
Model output is probabilistic text, and the code that consumes it usually assumes a shape. When the assumption is implicit, violations surface as exceptions deep in a pipeline or, worse, as plausible wrong values. An explicit contract moves the assumption into a schema and a validation step, so a malformed response becomes a handled case with a defined outcome rather than an incident.
Mental model
An output contract is a delivery specification: dimensions, packaging, and what the receiver does with a damaged box. The comparison breaks because the sender can produce a box that meets every dimension and still contains the wrong item. Inspection has to check contents, not just the crate.
How it works
Contract budget · illustrative tokens
2,000 input+ 700 reserve= 2,700 of 3,000
300 free
Step 1Contract in prose
The four fields cost 480 of the 2,000 input tokens: nearly a quarter of the prompt describes the shape of an answer that has not been generated yet.
Output reserve
700480 of 2,000 input tokens are the contract; 1,220 are evidence and request.
Pick a section to read its token count, its role and its state.
Figure. A contract is paid in input tokens before generation and in reserve if it is enforced. Writing the four fields as a schema costs 160 tokens instead of 480; a repair call that re-sends the prompt, the invalid object and the error costs 2,730 input tokens, 430 past the window.
What to notice. Counts are authored for a 3,000-token window with a 700-token reserve; no response was generated and no validator was run. The observed output length the reserve should be sized to is a p99 in real systems and is only named here. Provider-enforced schema modes are not drawn; the schema block stands for whatever the request carries to constrain the shape.
Text version
A 3,000-token window with a 700-token answer reserve, so 2,300 tokens are available. The default first step describes the contract in prose: instructions 300, field descriptions 480, document 1,100 and question 120, for 2,000 input tokens and 300 free. The second step replaces the prose with a 160-token schema: 1,680 tokens and 620 free. The third step spends the freed 320 tokens on a longer document, returning to 2,000 tokens and 300 free. The fourth step cuts the reserve from 700 to 300 tokens: the available prompt space rises to 2,700 and 700 tokens look free, while the longest valid object needs 640 and is now truncated. The fifth step is a repair call: instructions, schema, the longer document, the question, a 640-token invalid object and a 90-token validator error, for 2,730 tokens — 430 over the window. Selecting a section shows its count and role.
- Write the contract from the consumer's needs: required fields, types, allowed values and nullability.
- Express it as a schema the provider can enforce, or as a validation step after generation.
- Define the failure path for each violation class: retry, repair with a second call, degrade or reject.
- Add semantic checks the schema cannot express, such as totals reconciling or ids existing.
- Test with adversarial outputs: truncated JSON, extra fields, wrong enum, empty arrays.
- Version the contract with the prompt, since a field change is a breaking interface change.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Schema-enforced generation | Parseable by construction | Model and provider constraints, less flexibility | The task needs free-form prose |
| Prompt-only format request | Works anywhere | Occasional malformed output | Downstream code cannot tolerate repair |
| Retry on violation | Simple recovery | Extra latency and cost | Violations are systematic, not random |
| Repair pass | Keeps most of the content | Second call and possible drift | The violation is a missing fact |
Good for
Parseable by construction
Cost
Model and provider constraints, less flexibility
Fails when
The task needs free-form prose
- The task needs free-form prose
- Downstream code cannot tolerate repair
- Violations are systematic, not random
- The violation is a missing fact
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Schema-enforced generation is good for Parseable by construction, costs Model and provider constraints, less flexibility, and fails when The task needs free-form prose. Prompt-only format request is good for Works anywhere, costs Occasional malformed output, and fails when Downstream code cannot tolerate repair. Retry on violation is good for Simple recovery, costs Extra latency and cost, and fails when Violations are systematic, not random. Repair pass is good for Keeps most of the content, costs Second call and possible drift, and fails when The violation is a missing fact
Failure modes
- Truncated object. The response hits the token cap and the JSON is incomplete. Debug: check the finish reason and set the cap above the observed p99 output length.
- Schema-valid emptiness. Required fields are present but null or empty. Debug: assert non-empty values and cross-check against the input.
- Enum drift. The model returns a label outside the allowed set. Debug: log invalid values with the input that produced them and tighten the enum description.
In production
Track contract violation rate per route and model version, and alert when the repair path becomes the common path. Keep a fixture suite of malformed outputs to test the failure handling.
Common misconceptions
People think: A JSON schema makes the output correct.
Actually: It constrains structure; the values can still be wrong or unsupported.
People think: Validation belongs in the consumer.
Actually: Putting it at the boundary, with a defined failure path, keeps one consumer's assumption from becoming every consumer's bug.
Interview angle
Define the contract from the consumer side, then separate structural validation from semantic checks and give each a failure path. Strong answers mention truncation and versioning. Weak answers equate a schema with correctness and leave the retry or repair decision implicit.
Sources
- OpenAI, Structured outputs guide — schema-constrained responses and their limits.
- Kasai et al., Synchromesh — constrained generation that stays consistent with the input.
- Willard and Louf, Efficient Guided Generation for LLMs — grammar-constrained decoding with a schema index.