seniorReportedcurated
Users say a rate-limits answer quotes correct numbers but never says what they apply to. You trace it to one chunk. Diagnose and fix.
30-second answer
- This is structure loss: a fixed-window split cut the table away from the sentence introducing it, so the model reported numbers with no idea what they applied to.
- Confirm by printing the retrieved chunk verbatim, because candidates look plausible until you read them cold.
- Fix at ingest: keep tables atomic, serialise each row with its column headers, and prepend a heading trail plus a one-line contextual summary.
- Then add a golden question whose answer requires the table, so the regression is caught by the suite rather than by users.
- An atomic table can exceed the embedding model's maximum input, so summarise it into a retrievable description with the full table as the parent payload, and cut on row groups only when you must.
Likely follow-ups
Open a question to rehearse the answer.
How would you detect this class of context loss before a user reports it, without reading every chunk?
- Scan the parsed structure for table regions with no heading attached, which is a mechanical check rather than a judgement call.
- Track the share of chunks whose first line has an unresolved pronoun or no subject, the signature of an orphaned heading.
- Run a probe set of questions that require a table and alert on recall, since that is the subset where this failure surfaces.
What does the contextual summary cost per document, and how do you keep it from bloating the index?
- It is one model call per chunk at ingest, a real bill on a large corpus but a one-off per document version.
- Keep the summary to a sentence and cache it keyed by chunk ID and document version, so retries do not regenerate it.
- Store original text for citation and contextualised text for embedding, which keeps prompt tokens from growing with the index.
If keeping the table atomic exceeds the embedding model's input limit, what do you do instead of splitting it?
- Summarise the table into a retrievable description and keep the full table as the parent payload the model reads.
- If you must cut it, split on row groups with the header repeated on each part, so no part loses its column meaning.
- Assert chunk token length against the model maximum at ingest and fail the job rather than truncating silently.
Red flags
- rewrites the grounding prompt first
- never inspects retrieved text
Concepts this tests
Tags
Source: algorithmine.com
More in Data & storage
- How do you choose a chunk size, and what concretely goes wrong at each extreme?
- How do you ingest a 200-page PDF of charts and tables where the answer exists only inside a line chart, with no matching text?
- Page 1 of a financial report says "all amounts in thousands". How do you stop chunks from losing that document-wide context?