Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 45 of 57

Multi-hop

0 of 57 complete

Why it matters

Some questions cannot be answered from any single passage: 'which subsidiary of the company that makes X filed the lawsuit' requires finding the maker, then the subsidiary, then the filing. A single retrieval call returns passages about one of those, and the model fills the rest from memory. Multi-hop retrieval makes each link explicit so the chain can be checked.

Mental model

Multi-hop retrieval is following footnotes: each source points to the next, and the answer is at the end of the chain. The metaphor breaks because a footnote is authoritative, while the bridge entity here is extracted by a model and can be the wrong entity entirely.

How it works

Mechanism, step by stepInteractive
  1. 1Detect that the question needs a chain: it names an entity that must be resolved before the target attribute can be looked up.
  2. 2Retrieve for the first hop and extract the bridge entity from the returned passages, with the supporting span.
  3. 3Form the next query from the bridge entity and the remaining part of the question, not from the full original.
  4. 4Carry forward which passages support the chain so the final answer can cite every link.
  5. 5Stop when the target attribute is found or when a hop returns no new entity; then answer or report the break.
  6. 6Bound hops and total retrieval calls, and record the chain for every multi-hop request.

Step 1 of 6

Detect that the question needs a chain: it names an entity that must be resolved before the target attribute can be looked up.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Detect that the question needs a chain: it names an entity that must be resolved before t… 2. Retrieve for the first hop and extract the bridge entity from the returned passages, with… 3. Form the next query from the bridge entity and the remaining part of the question, not fr… 4. Carry forward which passages support the chain so the final answer can cite every link. 5. Stop when the target attribute is found or when a hop returns no new entity; then answer … 6. Bound hops and total retrieval calls, and record the chain for every multi-hop request.

  1. Detect that the question needs a chain: it names an entity that must be resolved before the target attribute can be looked up.
  2. Retrieve for the first hop and extract the bridge entity from the returned passages, with the supporting span.
  3. Form the next query from the bridge entity and the remaining part of the question, not from the full original.
  4. Carry forward which passages support the chain so the final answer can cite every link.
  5. Stop when the target attribute is found or when a hop returns no new entity; then answer or report the break.
  6. Bound hops and total retrieval calls, and record the chain for every multi-hop request.

Trade-offs

ChoiceGood forCostFails when
Sequential hopsCorrect on genuine chainsLatency grows with depthThe question has many links
Parallel fan-outFaster on independent sub-questionsCannot resolve dependent linksThe second hop needs the first answer
Single broad retrievalOne call and simpleMisses the second documentEvidence spans documents
Model-generated bridgesHandles free-form phrasingExtraction errors propagateEntities are ambiguous or numeric
Trade-off mapInteractive

Good for

Correct on genuine chains

Cost

Latency grows with depth

Fails when

The question has many links

  1. The question has many links
  2. The second hop needs the first answer
  3. Evidence spans documents
  4. Entities are ambiguous or numeric

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Sequential hops is good for Correct on genuine chains, costs Latency grows with depth, and fails when The question has many links. Parallel fan-out is good for Faster on independent sub-questions, costs Cannot resolve dependent links, and fails when The second hop needs the first answer. Single broad retrieval is good for One call and simple, costs Misses the second document, and fails when Evidence spans documents. Model-generated bridges is good for Handles free-form phrasing, costs Extraction errors propagate, and fails when Entities are ambiguous or numeric

Failure modes

  • Wrong bridge entity. The first hop returns a similarly named company and the chain follows the wrong branch. Debug: print the extracted bridge with its supporting span and check it against the question's entity.
  • Chain never terminates. Each hop surfaces a new entity and the loop keeps going. Debug: log hop count and the entity added per hop, and enforce a maximum.
  • Unsupported final answer. The answer cites only the last passage and the earlier links are unverifiable. Debug: require a citation per hop and check each link against its own source.

In production

Track hops per request, chain completion rate and the share of chains that end without the target attribute. Alert when mean hops rises, which usually signals a retrieval regression at hop one.

Common misconceptions

People think: Multi-hop is just top-k with a bigger k.

Actually: Depth does not substitute for a second query conditioned on the first result.

People think: The model can fill in the missing hop from memory.

Actually: That produces an unverified link, which is the failure multi-hop retrieval exists to prevent.

Interview angle

Explain why the second query depends on the first answer, bound the hops, and keep a citation per link. Name bridge-entity error as the dominant failure. Retrieving more documents and letting the model reason across them is the answer that hides the chain.

Sources