Why it matters
Manual prompt iteration is slow and biased toward recent failures. Optimisation methods generate candidate prompts, score them on a set, and keep the best, sometimes using model-written critiques or gradients over instructions. The engineering problem is that the objective is a proxy: a prompt that wins on the metric may exploit judge quirks, overfit the sample or become impossible for a human to maintain.
Mental model
Prompt optimisation is a search over recipes judged by a tasting panel. The comparison breaks because the panel can be gamed, and the winning recipe may be excellent for the panel and strange for customers who were never in the room during judging.
How it works
Prompt budget · illustrative tokens
1,430 input+ 400 reserve= 1,830 of 2,000
170 free
Step 1Manual baseline
1,430 tokens of prompt and 170 free. It is readable, it has one copy of every section, and a reviewer can diff it.
Output reserve
400Baseline: five sections, 1,430 input tokens, 170 free.
Pick a section to read its token count, its role and its state.
Figure. A search does not find brevity; it finds whatever scores. Here the winning prompt is 530 tokens longer than the manual baseline and runs 360 over the window, and only a length constraint brings it back to 1,580 — leaving 20 tokens for the next incident's rule.
What to notice. Counts are authored for a 2,000-token window with a 400-token answer reserve; no optimiser, model or eval was run and no score is reported. The added blocks stand for a critique-derived rule, a restated instruction and duplicated examples — one plausible artefact set, not a measured one. The candidate budget, judge cost and the search set itself are not drawn.
Text version
A 2,000-token window with a 400-token answer reserve, so 1,600 tokens are available. The default first step is the manual baseline: instructions 300, task 80, examples 260, document 700 and question 90, for 1,430 input tokens and 170 free. The second step is the unconstrained winner: the same blocks plus a critique-derived rule 150, a restated instruction 160 and duplicated examples 220, for 1,960 tokens — 360 over the window, with the cells crossing the cap rule. The third step applies a length constraint that drops the restatement and the duplicated examples, leaving 1,580 tokens and 20 free. The fourth step adds one more 180-token rule from a new failure: 1,760 tokens, 160 over. Selecting a section shows its count and role.
- Fix the evaluation set, the metric and the acceptance threshold before generating any candidate.
- Split into a search set and a held-out set, and optimise only on the search set.
- Choose the search method: instruction rewriting with critiques, candidate sampling, or a compiled pipeline with learned prompts.
- Constrain the search: maximum length, required sections, forbidden content and provider portability.
- Validate the winner on the held-out set and by human review of a sample, not just the metric.
- Keep the previous prompt and compare production metrics after rollout.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Manual iteration | Readable, fast for small changes | Slow and biased | The task has many interacting rules |
| Automated search | Finds non-obvious prompts | Metric overfitting and odd artefacts | The evaluation set is small |
| Compiled pipelines | Joint optimisation of prompts and flow | Framework lock-in and opaque prompts | The team must debug prompts by hand |
| Large candidate budget | Better score on the search set | Cost and diminishing returns | The held-out set is noisy |
Good for
Readable, fast for small changes
Cost
Slow and biased
Fails when
The task has many interacting rules
- The task has many interacting rules
- The evaluation set is small
- The team must debug prompts by hand
- The held-out set is noisy
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Manual iteration is good for Readable, fast for small changes, costs Slow and biased, and fails when The task has many interacting rules. Automated search is good for Finds non-obvious prompts, costs Metric overfitting and odd artefacts, and fails when The evaluation set is small. Compiled pipelines is good for Joint optimisation of prompts and flow, costs Framework lock-in and opaque prompts, and fails when The team must debug prompts by hand. Large candidate budget is good for Better score on the search set, costs Cost and diminishing returns, and fails when The held-out set is noisy
Failure modes
- Metric gaming. The prompt exploits judge preferences and fails human review. Debug: compare metric gains with human ratings on a sample of the winner's outputs.
- Search overfitting. The winner beats the baseline on the search set and loses on held-out data. Debug: report both scores and the gap between them.
- Unmaintainable prompt. The optimised prompt is long, redundant and cannot be edited safely. Debug: measure length and section count against the manual baseline before adopting.
In production
Run optimised prompts through the same canary and monitoring as hand-written ones, and keep the metric's human agreement under review. Re-optimise only when the task or model changes materially.
Common misconceptions
People think: Optimisation removes the need for evaluation design.
Actually: It depends entirely on the metric; a bad metric produces a confidently bad prompt.
People think: An optimised prompt is optimal.
Actually: It is the best candidate found under a search budget against a proxy, on a particular sample.
Interview angle
Treat optimisation as search under a proxy objective and spend the answer on the metric, the splits and the constraints. Strong answers mention human review of the winner and maintenance cost. Weak answers claim an optimiser finds the best prompt without saying what it was measured against.
Sources
- Zhou et al., Large Language Models Are Human-Level Prompt Engineers — automatic instruction search and its evaluation.
- Pryzant et al., Automatic Prompt Optimization with Gradient Descent and Beam Search — iterative critique-driven prompt edits.
- Khattab et al., DSPy — compiling pipelines and prompts against a metric.