Why it matters
Chunking removes exactly the information that makes a sentence interpretable: the heading, the entity, the section. A dense vector built from the fragment encodes the ambiguity, so a query naming the real subject does not match. Contextual retrieval restores the missing frame by writing a few sentences that situate the chunk in its document and embedding that alongside the text.
Mental model
It is annotating each page of a photocopied report with a margin note saying which chapter it came from. The note makes the page findable. The metaphor breaks because the note is generated, so it can assert a subject the page does not actually discuss.
How it works
- 1Parse the document and keep its title and outline available to the context generator.
- 2For each chunk, generate a short situating passage that names the document, the section and the entity the chunk refers to.
- 3Prepend that passage to the chunk text and embed the concatenation, caching the generated context per chunk.
- 4Store the original text separately so citations quote the source and not the generated frame.
- 5Apply the same contextualisation to the lexical index if you run hybrid retrieval, since identifiers benefit too.
- 6Evaluate recall with and without context on queries that use the document's subject rather than the chunk's wording.
Step 1 of 6
Parse the document and keep its title and outline available to the context generator.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Parse the document and keep its title and outline available to the context generator. 2. For each chunk, generate a short situating passage that names the document, the section a… 3. Prepend that passage to the chunk text and embed the concatenation, caching the generated… 4. Store the original text separately so citations quote the source and not the generated fr… 5. Apply the same contextualisation to the lexical index if you run hybrid retrieval, since … 6. Evaluate recall with and without context on queries that use the document's subject rathe…
- Parse the document and keep its title and outline available to the context generator.
- For each chunk, generate a short situating passage that names the document, the section and the entity the chunk refers to.
- Prepend that passage to the chunk text and embed the concatenation, caching the generated context per chunk.
- Store the original text separately so citations quote the source and not the generated frame.
- Apply the same contextualisation to the lexical index if you run hybrid retrieval, since identifiers benefit too.
- Evaluate recall with and without context on queries that use the document's subject rather than the chunk's wording.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Generated context per chunk | Recovers subject and pronoun references | One model call per chunk at ingest | The corpus is large and ingest budget is fixed |
| Structural context only | Deterministic and free | Cannot resolve references the structure does not encode | Documents use implicit subjects throughout |
| Context on children only | Cheap and targeted | Parent lookups still lack framing | The pipeline returns raw parents |
| No contextualisation | Simplest ingest | Fragments embed as ambiguous statements | Chunks are short and pronoun-heavy |
Good for
Recovers subject and pronoun references
Cost
One model call per chunk at ingest
Fails when
The corpus is large and ingest budget is fixed
- The corpus is large and ingest budget is fixed
- Documents use implicit subjects throughout
- The pipeline returns raw parents
- Chunks are short and pronoun-heavy
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Generated context per chunk is good for Recovers subject and pronoun references, costs One model call per chunk at ingest, and fails when The corpus is large and ingest budget is fixed. Structural context only is good for Deterministic and free, costs Cannot resolve references the structure does not encode, and fails when Documents use implicit subjects throughout. Context on children only is good for Cheap and targeted, costs Parent lookups still lack framing, and fails when The pipeline returns raw parents. No contextualisation is good for Simplest ingest, costs Fragments embed as ambiguous statements, and fails when Chunks are short and pronoun-heavy
Failure modes
- Context contradicts the chunk. The generator invents a subject and the chunk becomes retrievable for the wrong topic. Debug: sample generated contexts and check each names an entity that appears in the parent document.
- Context dominates the vector. The prepended frame is longer than the chunk, so all chunks of a document become near-duplicates. Debug: compare pairwise similarity within one document before and after contextualisation.
- Context not cached. A retry regenerates different context and produces vectors that no longer match the index. Debug: hash the generated context per chunk and confirm it is stable across runs.
In production
Cache generated context keyed by chunk id and document version, and rebuild it when the document changes. Track ingest cost per document and the similarity spread within a document, which collapses if context overwhelms the text.
Common misconceptions
People think: Contextual retrieval is just adding the title to every chunk.
Actually: The value comes from resolving the specific references in that chunk, which a title alone does not do.
People think: It replaces the need for good chunk boundaries.
Actually: It compensates for fragments; chunks that mix topics still produce diluted vectors.
Interview angle
Explain the ambiguity chunking introduces and why the fix belongs at index time. Keep generated context separate from stored source text for citation, and mention caching. Prepending the document title and stopping there misses the specific reference resolution that gives the technique its value.
Sources
- Anthropic, Contextual Retrieval — prepending chunk context before embedding and lexical indexing.
- Robertson & Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond — why added context also changes lexical term statistics.
- PostgreSQL, Full Text Search — the lexical index that also benefits from contextualised chunks.