Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 30 of 57

Query decomposition

0 of 57 complete

Why it matters

A question like 'which plan is cheaper than the A2 and includes SSO' contains a constraint and a comparison, and its embedding lands between them. Retrieval then returns passages about pricing or about SSO but rarely both. Decomposition makes each need explicit so each is matched against the corpus, at the cost of more retrieval calls and a synthesis step that must reconcile the results.

Mental model

Decomposition is splitting a shopping list into separate aisles instead of wandering the shop hoping to pass everything. The metaphor breaks when an item's aisle depends on what you find in another, which is why some lists must be walked in order.

How it works

Mechanism, step by stepInteractive
  1. 1Classify the question: single need, independent parts, or dependent chain.
  2. 2For independent parts, emit sub-queries in one pass and retrieve them in parallel under a shared budget.
  3. 3For dependent chains, retrieve the first hop, extract the bridging entity, then form the next query from it.
  4. 4Attribute each retrieved passage to the sub-query it answers, so synthesis can detect a missing part.
  5. 5Cap the number of sub-queries and the total retrieved passages, and stop early when a part returns nothing usable.
  6. 6Answer from the attributed evidence, and state which part could not be supported rather than merging silently.

Step 1 of 6

Classify the question: single need, independent parts, or dependent chain.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Classify the question: single need, independent parts, or dependent chain. 2. For independent parts, emit sub-queries in one pass and retrieve them in parallel under a… 3. For dependent chains, retrieve the first hop, extract the bridging entity, then form the … 4. Attribute each retrieved passage to the sub-query it answers, so synthesis can detect a m… 5. Cap the number of sub-queries and the total retrieved passages, and stop early when a par… 6. Answer from the attributed evidence, and state which part could not be supported rather t…

  1. Classify the question: single need, independent parts, or dependent chain.
  2. For independent parts, emit sub-queries in one pass and retrieve them in parallel under a shared budget.
  3. For dependent chains, retrieve the first hop, extract the bridging entity, then form the next query from it.
  4. Attribute each retrieved passage to the sub-query it answers, so synthesis can detect a missing part.
  5. Cap the number of sub-queries and the total retrieved passages, and stop early when a part returns nothing usable.
  6. Answer from the attributed evidence, and state which part could not be supported rather than merging silently.

Trade-offs

ChoiceGood forCostFails when
Parallel decompositionCheap, bounded and easy to cacheCannot resolve dependent referencesThe second part depends on the first answer
Sequential decompositionHandles chains and comparisonsLatency multiplies with hopsThe question has many dependent steps
Single queryOne retrieval callOne vector for several needsQuestions combine constraints
Model-driven decompositionHandles free-form phrasingDecomposition errors are hard to seeThe domain has rigid query templates
Trade-off mapInteractive

Good for

Cheap, bounded and easy to cache

Cost

Cannot resolve dependent references

Fails when

The second part depends on the first answer

  1. The second part depends on the first answer
  2. The question has many dependent steps
  3. Questions combine constraints
  4. The domain has rigid query templates

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Parallel decomposition is good for Cheap, bounded and easy to cache, costs Cannot resolve dependent references, and fails when The second part depends on the first answer. Sequential decomposition is good for Handles chains and comparisons, costs Latency multiplies with hops, and fails when The question has many dependent steps. Single query is good for One retrieval call, costs One vector for several needs, and fails when Questions combine constraints. Model-driven decomposition is good for Handles free-form phrasing, costs Decomposition errors are hard to see, and fails when The domain has rigid query templates

Failure modes

  • Wrong split. A qualifier is separated from the entity it modifies, so each sub-query retrieves the wrong thing. Debug: read the sub-queries and check that every modifier stays with its noun.
  • Missing part in synthesis. One sub-query returns nothing and the answer silently omits that constraint. Debug: require one attributed passage per sub-query before answering, and report gaps.
  • Hop explosion. A chain keeps generating new sub-queries and latency grows without a bound. Debug: log hops per request and enforce a hard maximum.

In production

Track sub-queries per request, per-part retrieval success and total latency. Alert when the share of questions with an unanswered part rises, which usually means the decomposition drifted.

Common misconceptions

People think: Decomposition always improves multi-hop accuracy.

Actually: It helps when the split matches the question's structure and hurts when it severs a qualifier from its subject.

People think: More sub-queries mean better coverage.

Actually: Each adds retrieval cost and noise; coverage is bounded by whether the parts are genuinely independent.

Interview angle

Classify the question before splitting, distinguish parallel from sequential, and bound the hops. Attribute evidence per part and report unsupported parts. Splitting with a model and concatenating whatever comes back is the answer that produces confident partial answers.

Sources