midReportedcurated
You stuff 60 retrieved documents into one prompt and the model uses facts from the beginning and end but ignores a crucial fact in the middle. Diagnose it and fix it.
30-second answer
- Long-context models use information unevenly: facts at the start and end are recalled more reliably than facts buried in the middle, and accuracy degrades as irrelevant context grows.
- Sixty documents also means the decisive passage competes with fifty-nine distractors, so the fix is curation rather than a bigger window.
- Retrieve and rerank fewer chunks, put the most relevant evidence at the edges, keep instructions close to the question, and measure recall by position on your own tasks.
- Aggressive pruning can drop a needed fact, so pair it with citation checks and an abstain path rather than assuming fewer documents is always better.
- Lost in the middle is a behavioural bias, not a hard capacity limit, so a larger context window does not fix it and can make it more expensive.
Likely follow-ups
Open a question to rehearse the answer.
How would you measure position sensitivity on your own task?
- Take queries with a known supporting passage, place it at several positions among the same distractors, and measure accuracy per position bucket.
- Hold retrieval fixed and change only order, or the result mixes ranking quality with position sensitivity.
- Use enough queries per bucket to separate a real curve from noise, and report the worst bucket rather than the mean.
When is a larger context window genuinely better than retrieval?
- It wins for one-off analysis of a large artefact where the whole document must be considered together and requests do not repeat.
- It also wins when cross-document reasoning needs many passages and retrieval recall is already high enough that pruning only removes evidence.
- At volume, cost, latency and middle neglect usually make retrieval plus reranking the better default.
Red flags
- Recommends only switching to a larger-context model
- Raises temperature to make the model explore the context
Concepts this tests
Tags
Source: github.com
More in Prompting & context engineering
- An agent reads retrieved documents and tool outputs and can take actions. How do you defend against indirect prompt injection, and why can't a better system prompt solve it?
- Your JSON parsing works in development but fails intermittently in production even at temperature 0. Diagnose it and describe the durable fix.
- Your chain-of-thought prompt is not improving accuracy on a reasoning task. What do you check and what do you change?