Why it matters
Some questions cannot be answered from any single passage: 'which subsidiary of the company that makes X filed the lawsuit' requires finding the maker, then the subsidiary, then the filing. A single retrieval call returns passages about one of those, and the model fills the rest from memory. Multi-hop retrieval makes each link explicit so the chain can be checked.
Mental model
Multi-hop retrieval is following footnotes: each source points to the next, and the answer is at the end of the chain. The metaphor breaks because a footnote is authoritative, while the bridge entity here is extracted by a model and can be the wrong entity entirely.
How it works
- 1Detect that the question needs a chain: it names an entity that must be resolved before the target attribute can be looked up.
- 2Retrieve for the first hop and extract the bridge entity from the returned passages, with the supporting span.
- 3Form the next query from the bridge entity and the remaining part of the question, not from the full original.
- 4Carry forward which passages support the chain so the final answer can cite every link.
- 5Stop when the target attribute is found or when a hop returns no new entity; then answer or report the break.
- 6Bound hops and total retrieval calls, and record the chain for every multi-hop request.
Step 1 of 6
Detect that the question needs a chain: it names an entity that must be resolved before the target attribute can be looked up.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Detect that the question needs a chain: it names an entity that must be resolved before t… 2. Retrieve for the first hop and extract the bridge entity from the returned passages, with… 3. Form the next query from the bridge entity and the remaining part of the question, not fr… 4. Carry forward which passages support the chain so the final answer can cite every link. 5. Stop when the target attribute is found or when a hop returns no new entity; then answer … 6. Bound hops and total retrieval calls, and record the chain for every multi-hop request.
- Detect that the question needs a chain: it names an entity that must be resolved before the target attribute can be looked up.
- Retrieve for the first hop and extract the bridge entity from the returned passages, with the supporting span.
- Form the next query from the bridge entity and the remaining part of the question, not from the full original.
- Carry forward which passages support the chain so the final answer can cite every link.
- Stop when the target attribute is found or when a hop returns no new entity; then answer or report the break.
- Bound hops and total retrieval calls, and record the chain for every multi-hop request.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Sequential hops | Correct on genuine chains | Latency grows with depth | The question has many links |
| Parallel fan-out | Faster on independent sub-questions | Cannot resolve dependent links | The second hop needs the first answer |
| Single broad retrieval | One call and simple | Misses the second document | Evidence spans documents |
| Model-generated bridges | Handles free-form phrasing | Extraction errors propagate | Entities are ambiguous or numeric |
Good for
Correct on genuine chains
Cost
Latency grows with depth
Fails when
The question has many links
- The question has many links
- The second hop needs the first answer
- Evidence spans documents
- Entities are ambiguous or numeric
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Sequential hops is good for Correct on genuine chains, costs Latency grows with depth, and fails when The question has many links. Parallel fan-out is good for Faster on independent sub-questions, costs Cannot resolve dependent links, and fails when The second hop needs the first answer. Single broad retrieval is good for One call and simple, costs Misses the second document, and fails when Evidence spans documents. Model-generated bridges is good for Handles free-form phrasing, costs Extraction errors propagate, and fails when Entities are ambiguous or numeric
Failure modes
- Wrong bridge entity. The first hop returns a similarly named company and the chain follows the wrong branch. Debug: print the extracted bridge with its supporting span and check it against the question's entity.
- Chain never terminates. Each hop surfaces a new entity and the loop keeps going. Debug: log hop count and the entity added per hop, and enforce a maximum.
- Unsupported final answer. The answer cites only the last passage and the earlier links are unverifiable. Debug: require a citation per hop and check each link against its own source.
In production
Track hops per request, chain completion rate and the share of chains that end without the target attribute. Alert when mean hops rises, which usually signals a retrieval regression at hop one.
Common misconceptions
People think: Multi-hop is just top-k with a bigger k.
Actually: Depth does not substitute for a second query conditioned on the first result.
People think: The model can fill in the missing hop from memory.
Actually: That produces an unverified link, which is the failure multi-hop retrieval exists to prevent.
Interview angle
Explain why the second query depends on the first answer, bound the hops, and keep a citation per link. Name bridge-entity error as the dominant failure. Retrieving more documents and letting the model reason across them is the answer that hides the chain.
Sources
- Yang et al., HotpotQA — multi-hop questions requiring evidence from several documents.
- Trivedi et al., Interleaving Retrieval with Chain-of-Thought Reasoning — retrieving at each reasoning step rather than once up front.
- Press et al., Measuring and Narrowing the Compositionality Gap — self-ask follow-up questions that expose the hops.