Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 17 of 37

Observation design

0 of 37 complete

Why it matters

A tool can return a correct result that still causes a wrong answer: a two-hundred-row table where the relevant row is last, a document cut mid-sentence, an error page rendered as content. The observation is the interface between deterministic code and a probabilistic reader, so its size, order and framing directly affect accuracy and cost.

Mental model

An observation is a briefing note, not a data dump. The metaphor breaks because a briefer knows the reader's question and can omit safely, while the tool often does not, so filtering must be conservative and every omission visible.

How it works

Shape, rank, cap with a marker, then hand the model a bounded readInteractive

Keyed transitions

  1. Cap + marker → Model step: omitted count ∧ expand id
  2. Fits? → Ambiguous empty: ∅ as list
transitionfailurerepeat

Event · step 1 of 9

Tool returns 214 rows

Entry: a query returns 214 rows with navigation, headers and a four-line footer. Nothing is shaped yet; the raw payload is 9,800 tokens, and most of it is chrome.

In state

  • Tool result

Next: Shape chosen

Tokens
9.8k
Omitted
0
Marks
0
Step 1/9 · Tool returns 214 rows

Figure. Nine states. Shape chooses a ranked list, a record or a reference; Rank orders rows before Fits cuts, and Cap is the state where an omission becomes visible. Model step is terminal because it reads a marked, bounded observation. The red states are the page's failures: an empty result shaped like a list, a cut with no marker, and boilerplate left in.

What to notice. The 1,500-token cap, the top-8 cut, the 214 rows and the 208 omitted are illustrative. The full raw payload and the summarised variant share Shape but are not drawn; a summary that rewords ids is a distinct failure the page covers in prose. The reference-and-expand shape shares Model step.

Text version

A state machine with nine states. Tool result is the start, a pill. Tool result goes to Shape, a hexagonal decision, on raw. Shape goes to Rank on ranked list, and to Boilerplate flood, a red failure box, on navigation kept. Rank goes to Fits on top 8. Fits goes to Cap plus marker, a checkpoint, on over cap, to Model step, a terminal pill, when the observation is at most 1.5k tokens, and to Ambiguous empty, a red failure box, when an empty result is shaped as a list. Cap plus marker goes to Model step on a marker with the omitted count, and to Silent cut, a red failure box, on no marker. The trace walks Tool result, Shape, Rank, Fits, Cap plus marker, Model step, then Ambiguous empty, then Silent cut, then Boilerplate flood. Counters show observation tokens falling from 9.8k to 1.4k, omitted rows at 208, and marks at 1 only after Cap.

  1. Decide the shape per tool: a compact record, a ranked list, or a reference the agent can expand.
  2. Cap size with an explicit marker stating how much was omitted and how to get the rest.
  3. Put the most decision-relevant fields first and keep exact identifiers verbatim.
  4. Strip boilerplate such as navigation, headers and repeated disclaimers before it reaches the model.
  5. Give empty, error and partial results distinct structures.
  6. Log the exact observation text with the step so a bad decision can be attributed.

Trade-offs

ChoiceGood forCostFails when
Full raw outputNo information lossToken cost and attention dilutionOutputs are large or repetitive
Summarised outputCheap and focusedSummariser errors and lost idsExact values matter
Ranked top-kRelevant and shortMisses the tailThe answer is in an unusual record
Reference and expandBounded context, drill-down possibleExtra turn and latencyThe agent rarely needs the detail
Trade-off mapInteractive

Good for

No information loss

Cost

Token cost and attention dilution

Fails when

Outputs are large or repetitive

  1. Outputs are large or repetitive
  2. Exact values matter
  3. The answer is in an unusual record
  4. The agent rarely needs the detail

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Full raw output is good for No information loss, costs Token cost and attention dilution, and fails when Outputs are large or repetitive. Summarised output is good for Cheap and focused, costs Summariser errors and lost ids, and fails when Exact values matter. Ranked top-k is good for Relevant and short, costs Misses the tail, and fails when The answer is in an unusual record. Reference and expand is good for Bounded context, drill-down possible, costs Extra turn and latency, and fails when The agent rarely needs the detail

Failure modes

  • Silent truncation. The model treats a cut list as complete and reports a wrong total. Debug: assert every truncated observation carries an explicit marker and a count.
  • Boilerplate flood. Navigation text dominates the observation and the answer quotes it. Debug: compare observation length with extracted content length per tool.
  • Ambiguous empty. No results and a failed call look identical. Debug: check the observation schema has distinct states for empty, error and partial.

In production

Track observation tokens per tool, truncation rate and the share of observations the agent quotes. Alert when a tool's average observation size jumps after a deploy.

Common misconceptions

People think: Returning everything is the safest choice.

Actually: Unfiltered output costs tokens and hides the relevant line; the safest is bounded output with visible omissions.

People think: Summarising is always safe.

Actually: A summary that drops an id or a number makes a correct tool result unusable.

Interview angle

Strong answers treat the observation as an interface with a budget, describe truncation markers and empty-versus-error states, and explain how observation quality is measured. Weak answers say the tool returns JSON.

Sources