seniorReportedcurated
Page 1 of a financial report says "all amounts in thousands". How do you stop chunks from losing that document-wide context?
30-second answer
- Treat document-level context as metadata rather than luck: store a document summary plus globals such as units, currency, fiscal period and table-of-contents path on every chunk.
- Prepend a one-sentence contextual header at embed time, naming the filing and the units, which fixes retrieval and lets the model scale numbers correctly.
- Contextual retrieval reports roughly 35-49% fewer failed retrievals for exactly this reason.
- The cost is an LLM call per chunk at ingest, about a dollar per million document tokens.
- It does not fix a unit change halfway through the document, so an appendix that switches to millions needs its own metadata rather than the document default.
Likely follow-ups
Open a question to rehearse the answer.
Do you embed the contextual header, prepend it only to the chunk body, or both, and why?
- Embed the header with the chunk text so document-level terms can retrieve it, and keep the original text for citation.
- Prepending only to the body changes what the model reads but not what the vector represents, which leaves retrieval unchanged.
- If you do both, check that the header does not dominate the vector and turn every chunk of a document into a near-duplicate.
When every chunk in a document shares the same boilerplate header, how do you stop it hurting discrimination?
- Measure pairwise similarity within one document before and after contextualisation, which collapses when the frame outweighs the text.
- Keep the header to one sentence naming the specific entity or section, not a generic restatement of the document title.
- Apply context where chunks are short and ambiguous, which is where it earns its ingest cost.
How do you propagate a mid-document unit change, like an appendix that switches to millions?
- Extract units per section at parse time and attach them to the chunk, so an appendix in millions carries its own scale.
- Where the unit is only stated in prose, have the contextual summary quote the sentence that changes it.
- Test with a question whose answer spans the change, since that is the case a document-level default silently gets wrong.
Red flags
- bigger chunks
- units only in prompt template
Concepts this tests
Tags
Source: adilshamim8.medium.com
More in Data & storage
- How do you choose a chunk size, and what concretely goes wrong at each extreme?
- Users say a rate-limits answer quotes correct numbers but never says what they apply to. You trace it to one chunk. Diagnose and fix.
- How do you ingest a 200-page PDF of charts and tables where the answer exists only inside a line chart, with no matching text?