Skip to content
AI Engineer Guide.
midReportedcurated

You stuff 60 retrieved documents into one prompt and the model uses facts from the beginning and end but ignores a crucial fact in the middle. Diagnose it and fix it.

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

How would you measure position sensitivity on your own task?
  • Take queries with a known supporting passage, place it at several positions among the same distractors, and measure accuracy per position bucket.
  • Hold retrieval fixed and change only order, or the result mixes ranking quality with position sensitivity.
  • Use enough queries per bucket to separate a real curve from noise, and report the worst bucket rather than the mean.
When is a larger context window genuinely better than retrieval?
  • It wins for one-off analysis of a large artefact where the whole document must be considered together and requests do not repeat.
  • It also wins when cross-document reasoning needs many passages and retrieval recall is already high enough that pruning only removes evidence.
  • At volume, cost, latency and middle neglect usually make retrieval plus reranking the better default.

Red flags

  • Recommends only switching to a larger-context model
  • Raises temperature to make the model explore the context

Concepts this tests

Tags

Source: github.com

More in Prompting & context engineering