Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 9 of 26

Decomposition

0 of 26 complete

Why it matters

A prompt that extracts, reasons and formats in one pass fails in ways that are hard to diagnose: a bad extraction and a bad inference look the same in the output. Decomposition gives each stage a contract, so a failure can be traced to the stage that produced it. The cost is real: more calls, more latency, and intermediate representations that may drop the detail a later stage needs.

Mental model

Decomposition is an assembly line: each station does one operation and passes a part on. The metaphor breaks because a misread part is not visibly defective, and later stations will confidently build on it. The defect surfaces only in the finished product, far from its cause.

How it works

Three stages, three gates, one baselineInteractive
13 nodes · 4 branches
stagefilevalidation gaterootstop pointruled out or rejected

Step 1 of 7

One prompt fails unattributably

The first attempt is a single call: extract, total, format. It is exact on 13 of 25 invoices, and the failures do not say which of the three jobs went wrong.

branch rejected · current

single-prompt baseline

One call, 1.7s p50, correct on 13 of 25 invoices. The failures are unattributable: a bad span and a bad total look the same in the JSON.

Under Task · invoice to 3 fields, total, JSON

Step 1/7 · One prompt fails unattributably

Figure. A task split at its context and format boundaries: extraction, arithmetic and rendering, each with its own prompt and a gate on the hand-off. The single-prompt baseline it replaces is drawn as a sibling branch.

What to notice. Decomposition is an interface decision: every edge is a contract and every gate is a place a failure stops. The same edges carry the risk — a lossy hand-off and compounding error — which is why the pipeline is measured against the single prompt. Accuracies, latency and stage versions are illustrative; 0.95³ ≈ 0.86 is arithmetic on the stated per-stage rate, not a measurement.

Text version

A prompt pipeline drawn as a hierarchy. The root is an invoice task: extract three fields, compute a total and render JSON. Under it are three stages, each tagged with a 95 percent exact rate. Stage 1 extracts spans, with prompt extract.v3 at temperature zero and a gate that rejects any span not found verbatim in the text. Stage 2 computes the total, with prompt compute.v2 and a gate that requires the sum to be within one cent of the printed lines; a rejected child shows a summarised hand-off dropping the line amounts. Stage 3 renders JSON against schema v4 with a citation gate. Two further branches compare the single-prompt baseline, rejected at 52 percent exact match, with the three-stage result at 84 percent and three calls. A final step walks the comparison.

  1. Write the end-to-end task and mark the points where the required context or output format changes.
  2. Split only at those points, and define the input and output contract for each stage.
  3. Choose a representation that preserves what later stages need, such as verbatim spans rather than a summary.
  4. Test each stage against its own labelled examples before testing the pipeline.
  5. Add a validation gate between stages so a malformed intermediate result stops the chain.
  6. Measure the end-to-end score against a single-prompt baseline, including latency and cost.

Trade-offs

ChoiceGood forCostFails when
Single promptOne call, low latencyFailures are unattributableThe task mixes extraction, reasoning and formatting
Two or three stagesLocalised failures, checkable stepsHand-off loss and extra callsThe intermediate format cannot carry the detail
Many small stagesEach stage simple and testableCompounding error and orchestration costThe task is short and mostly linear
Deterministic code between stagesRemoves model error at that stepLess flexible parsingThe transformation needs judgement
Trade-off mapInteractive

Good for

One call, low latency

Cost

Failures are unattributable

Fails when

The task mixes extraction, reasoning and formatting

  1. The task mixes extraction, reasoning and formatting
  2. The intermediate format cannot carry the detail
  3. The task is short and mostly linear
  4. The transformation needs judgement

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Single prompt is good for One call, low latency, costs Failures are unattributable, and fails when The task mixes extraction, reasoning and formatting. Two or three stages is good for Localised failures, checkable steps, costs Hand-off loss and extra calls, and fails when The intermediate format cannot carry the detail. Many small stages is good for Each stage simple and testable, costs Compounding error and orchestration cost, and fails when The task is short and mostly linear. Deterministic code between stages is good for Removes model error at that step, costs Less flexible parsing, and fails when The transformation needs judgement

Failure modes

  • Hand-off loss. A summary drops the identifier the next stage needs. Debug: diff the intermediate output against the source spans and check the required fields are present.
  • Error compounding. Each stage is 95 percent accurate and the pipeline is far worse. Debug: measure per-stage accuracy and multiply, then compare with the observed end-to-end score.
  • Stage contract drift. A prompt change alters the intermediate format and breaks the next stage. Debug: validate the intermediate against a schema and fail the request, not the parser.

In production

Log per-stage inputs and outputs with a shared request id, and alert on per-stage validation failures rather than only end-to-end errors. Cap the number of stages so latency stays within budget.

Common misconceptions

People think: Decomposition always improves accuracy.

Actually: It helps when sub-tasks conflict, and hurts when hand-offs lose information or errors compound.

People think: More stages mean more control.

Actually: Each stage adds a contract to maintain and a place for silent corruption.

Interview angle

Start from the failure you are trying to isolate, then split only at context or format boundaries and define each contract. Strong answers mention per-stage evaluation and error compounding. Weak answers split everything into steps and never compare the pipeline with a single-prompt baseline.

Sources