Why it matters
A question like 'which plan is cheaper than the A2 and includes SSO' contains a constraint and a comparison, and its embedding lands between them. Retrieval then returns passages about pricing or about SSO but rarely both. Decomposition makes each need explicit so each is matched against the corpus, at the cost of more retrieval calls and a synthesis step that must reconcile the results.
Mental model
Decomposition is splitting a shopping list into separate aisles instead of wandering the shop hoping to pass everything. The metaphor breaks when an item's aisle depends on what you find in another, which is why some lists must be walked in order.
How it works
- 1Classify the question: single need, independent parts, or dependent chain.
- 2For independent parts, emit sub-queries in one pass and retrieve them in parallel under a shared budget.
- 3For dependent chains, retrieve the first hop, extract the bridging entity, then form the next query from it.
- 4Attribute each retrieved passage to the sub-query it answers, so synthesis can detect a missing part.
- 5Cap the number of sub-queries and the total retrieved passages, and stop early when a part returns nothing usable.
- 6Answer from the attributed evidence, and state which part could not be supported rather than merging silently.
Step 1 of 6
Classify the question: single need, independent parts, or dependent chain.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Classify the question: single need, independent parts, or dependent chain. 2. For independent parts, emit sub-queries in one pass and retrieve them in parallel under a… 3. For dependent chains, retrieve the first hop, extract the bridging entity, then form the … 4. Attribute each retrieved passage to the sub-query it answers, so synthesis can detect a m… 5. Cap the number of sub-queries and the total retrieved passages, and stop early when a par… 6. Answer from the attributed evidence, and state which part could not be supported rather t…
- Classify the question: single need, independent parts, or dependent chain.
- For independent parts, emit sub-queries in one pass and retrieve them in parallel under a shared budget.
- For dependent chains, retrieve the first hop, extract the bridging entity, then form the next query from it.
- Attribute each retrieved passage to the sub-query it answers, so synthesis can detect a missing part.
- Cap the number of sub-queries and the total retrieved passages, and stop early when a part returns nothing usable.
- Answer from the attributed evidence, and state which part could not be supported rather than merging silently.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Parallel decomposition | Cheap, bounded and easy to cache | Cannot resolve dependent references | The second part depends on the first answer |
| Sequential decomposition | Handles chains and comparisons | Latency multiplies with hops | The question has many dependent steps |
| Single query | One retrieval call | One vector for several needs | Questions combine constraints |
| Model-driven decomposition | Handles free-form phrasing | Decomposition errors are hard to see | The domain has rigid query templates |
Good for
Cheap, bounded and easy to cache
Cost
Cannot resolve dependent references
Fails when
The second part depends on the first answer
- The second part depends on the first answer
- The question has many dependent steps
- Questions combine constraints
- The domain has rigid query templates
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Parallel decomposition is good for Cheap, bounded and easy to cache, costs Cannot resolve dependent references, and fails when The second part depends on the first answer. Sequential decomposition is good for Handles chains and comparisons, costs Latency multiplies with hops, and fails when The question has many dependent steps. Single query is good for One retrieval call, costs One vector for several needs, and fails when Questions combine constraints. Model-driven decomposition is good for Handles free-form phrasing, costs Decomposition errors are hard to see, and fails when The domain has rigid query templates
Failure modes
- Wrong split. A qualifier is separated from the entity it modifies, so each sub-query retrieves the wrong thing. Debug: read the sub-queries and check that every modifier stays with its noun.
- Missing part in synthesis. One sub-query returns nothing and the answer silently omits that constraint. Debug: require one attributed passage per sub-query before answering, and report gaps.
- Hop explosion. A chain keeps generating new sub-queries and latency grows without a bound. Debug: log hops per request and enforce a hard maximum.
In production
Track sub-queries per request, per-part retrieval success and total latency. Alert when the share of questions with an unanswered part rises, which usually means the decomposition drifted.
Common misconceptions
People think: Decomposition always improves multi-hop accuracy.
Actually: It helps when the split matches the question's structure and hurts when it severs a qualifier from its subject.
People think: More sub-queries mean better coverage.
Actually: Each adds retrieval cost and noise; coverage is bounded by whether the parts are genuinely independent.
Interview angle
Classify the question before splitting, distinguish parallel from sequential, and bound the hops. Attribute evidence per part and report unsupported parts. Splitting with a model and concatenating whatever comes back is the answer that produces confident partial answers.
Sources
- Press et al., Measuring and Narrowing the Compositionality Gap — self-ask decomposition into follow-up sub-questions.
- Trivedi et al., Interleaving Retrieval with Chain-of-Thought Reasoning — sequential retrieval driven by intermediate reasoning steps.
- Zhou et al., Least-to-Most Prompting — solving a problem as an ordered sequence of easier sub-problems.