Why it matters
A prompt is a sequence, and attention over a long sequence is not uniform. Evidence buried in the middle of a large context is used less reliably than the same evidence at the edges, and instructions placed before a long document can be diluted by it. Ordering is a free intervention compared with retrieval or fine-tuning, but it is only testable if the prompt structure is explicit enough to permute.
Mental model
Context ordering is seating guests at a long table: the people near the host get heard. The comparison breaks because the model has no fixed host position, and the effective seats shift with the task and the model version, so the seating chart must be retested rather than assumed.
How it works
Prompt order · illustrative tokens
3,200 input+ 400 reserve= 3,600 of 3,600
window full
Step 1Order A — source order
The decisive passage is third of seven sections, behind 1,500 tokens of framing: 47–66% of the input, in the middle of the prompt. That is the stretch a long-context model uses least reliably.
Output reserve
400Pick a section to read its token count, its role and its state.
Figure. Seven sections and 3,200 tokens arranged two ways. The decisive passage spans 47–66% of the input in source order and 73–92% in relevance order. Attention curves are a labelled schematic, not measured weights.
What to notice. The curves are drawn, not computed. They stand for the well-documented tendency to use the start and end of a long input more reliably than the middle; the size of the effect depends on the model, the task and the input length. The figure makes no claim that reordering improves quality — only that it is cheap enough to test with a permutation.
Text version
A 3,600-token window with a 400-token output reserve, so the 3,200-token prompt fills it exactly. The default first step is Order A, source order: instructions 700, history 800, refund policy 600 marked as key evidence, escalation matrix 450, shipping table 400, question 100 and a format reminder 150. A readout under the ribbon says the key evidence spans 47–66% of the input, and a dashed schematic curve is high at the edges and low over that middle stretch. The second step shows Order B: the same seven sections, with the key evidence moved after the other evidence so it spans 73–92% of the input, and the readout changes to match. The third step stacks both orders on the same scale. Selecting a section shows its token count and role; the key evidence is marked in both orders.
- Fix the sections of the prompt so their order is a variable you can change deliberately.
- Place task instructions and hard constraints at the start, and repeat the output format at the end.
- Put the most relevant evidence closest to the question, and order the rest by relevance rather than by source order.
- Run an A/B on the same content in different orders and measure per-slice accuracy.
- Record the chosen order with the prompt version, since it is part of behaviour.
- Re-test after a model upgrade, because position sensitivity changes with training.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Instructions first | Clear framing, cacheable prefix | Can be diluted by long context after it | The document is very long |
| Instructions last | Recency effect on the final instruction | Model may have already formed an approach | The task requires early constraints |
| Relevance-ordered evidence | Best evidence near the question | Requires a relevance score | Scores are noisy |
| Source order | Preserves document structure | Buries the key passage | Documents are long and uneven |
Good for
Clear framing, cacheable prefix
Cost
Can be diluted by long context after it
Fails when
The document is very long
- The document is very long
- The task requires early constraints
- Scores are noisy
- Documents are long and uneven
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Instructions first is good for Clear framing, cacheable prefix, costs Can be diluted by long context after it, and fails when The document is very long. Instructions last is good for Recency effect on the final instruction, costs Model may have already formed an approach, and fails when The task requires early constraints. Relevance-ordered evidence is good for Best evidence near the question, costs Requires a relevance score, and fails when Scores are noisy. Source order is good for Preserves document structure, costs Buries the key passage, and fails when Documents are long and uneven
Failure modes
- Middle burial. The decisive passage sits in the middle and is ignored. Debug: move the gold passage to the start and end and compare answers.
- Instruction dilution. A constraint at the top is overridden by later text. Debug: repeat the constraint at the end and see whether compliance changes.
- Order-sensitive test. An evaluation compares two prompts that differ in order as well as content. Debug: hold content fixed and permute only the order.
In production
Keep ordering in the prompt version and include an order-permutation case in the test suite. Monitor answer quality when context length grows, since ordering effects worsen with length.
Common misconceptions
People think: Order does not matter if the content is the same.
Actually: Position changes attention and therefore accuracy, especially in long contexts.
People think: Putting everything at the end is safest.
Actually: Recency helps some instructions and harms evidence that needs framing before it.
Interview angle
Explain the positional effect, then describe a controlled permutation test rather than a hunch. Strong answers tie ordering to prompt versioning and model upgrades. Weak answers shuffle context or claim order is irrelevant once the content fits.
Sources
- Liu et al., Lost in the Middle — accuracy varies with the position of relevant context.
- Lu et al., Fantastically Ordered Prompts — prompt ordering changes task accuracy.
- Hsieh et al., RULER — effective context length and position-dependent failure.