Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 5 of 26

Few-shot

0 of 26 complete

Why it matters

Some tasks are hard to specify in prose and easy to demonstrate. A support ticket triage policy has a dozen boundary cases. An extraction schema has fields that are obvious once you see one filled example. Few-shot prompting is the cheapest way to communicate those cases without training a model.

It is also one of the few techniques that works across providers without special features. The mechanism is in-context learning: the model conditions on the demonstrations and continues the pattern. No gradients, no checkpoint, no deployment. That makes it attractive for fast iteration and dangerous as a permanent substitute for evaluation, because an example set is a dataset with all the usual dataset problems.

Mental model

Showing a new hire three finished tickets before asking them to handle the fourth. The examples convey the format, the level of detail and where the boundary sits.

The analogy breaks because the new hire builds a model of the task, while the language model predominantly matches patterns in the prompt. It can copy surface features you did not intend — a date format, a phrase, a label distribution — and it does not know which parts of the example are essential.

How it works

When examples are the fixInteractive
startquestionoutcomepath takenruled out
VerdictChoose· recommended hereConditional· with the named costStandard· the ordinary case

The rungs ask whether examples are the right fix, then how to select them; each edge carries the answer.

Step 1 of 5

Try the instruction first

If the format and the boundary are describable, a zero-shot baseline is the cheapest test and the control every example set must beat.

Condition · yes

Zero-shot baseline

Standard

Cheapest and often enough; measure per class before adding anything.

Keep it in the evaluation suite so every added example has to earn its tokens.

Path walked

  1. A task that is easier to show than to describe
  2. Can you state the format and boundary in prose?
  3. yesZero-shot baseline

Ruled out: Do you have examples of the format and one boundary case?, Collect examples first

Step 1/5 · Try the instruction first

Figure. Examples earn their tokens only when the instruction cannot carry the format or the boundary. The rungs separate a describable task from a missing example pool, a token bill that favours a fine-tune, and inputs too varied for a fixed set.

What to notice. The rungs follow the page's argument, not a measured ordering. The three-to-five count is the documentation's starting point, not a rule, and the token break-even depends on your prices and volume. The figure assumes the examples are representative; majority-label bias, format leakage and eval contamination are failures the diagram cannot show.

Text version

A decision ladder for choosing a prompting approach. A task that is easier to show than to describe starts rung 1: can you state the format and boundary in prose? Yes leads to a zero-shot baseline, measured per class. No continues to rung 2: do you have examples of the format and one boundary case? No leads to collecting examples first; yes continues. Rung 3 asks whether the token bill dominates at stable, high volume; yes leads to considering a fine-tune instead, because few-shot wins when the task changes often or volume is low. No continues to rung 4: do inputs vary widely in style or domain? Yes leads to retrieval-selected examples with one deliberately dissimilar boundary case; no leads to a fixed set of three to five, chosen by the held-out evaluation rather than by the count alone.

The evidence on what matters is useful. Min et al. found that replacing gold labels with random labels barely hurt classification accuracy in their experiments, while the label space, the input distribution and the format of the demonstrations mattered a great deal. That result is often misread as "labels don't matter". The correct reading is narrower: for classification-style tasks, the model is learning the format and the candidate set from the demonstrations, so a wrong label is less damaging than a wrong label space or an inconsistent format.

Liu et al. showed that example selection and ordering change accuracy, and that retrieving examples similar to the query outperforms a fixed set. That is the bridge to retrieval-selected examples: instead of a static k, select demonstrations per input from a pool.

Anthropic's documentation recommends around three to five examples for multishot prompting. Treat that as a starting point, not a rule; the right count is the one your eval set prefers.

Quick facts

Mechanism
In-context conditioning
Weights
Unchanged
Typical count
A handful
Main cost
Input tokens per call
Main risk
Label and selection bias
Best evidence
Held-out eval

The cost is linear and permanent

With kk demonstrations of average length ℓ\ell tokens, every request pays roughly k⋅ℓk \cdot \ell extra input tokens:

Tin≈Ttask+k⋅ℓ+TsystemT_{\text{in}} \approx T_{\text{task}} + k \cdot \ell + T_{\text{system}}

That cost recurs on every call, unlike a fine-tune which pays once. For a high-volume endpoint, the break-even point between few-shot and fine-tuning is a real calculation: few-shot wins when the task changes often or volume is low; fine-tuning wins when the format is stable and the token bill dominates.

Selecting and rendering examples

def render_examples(examples: list[dict]) -> str:
    blocks = []
    for ex in examples:
        blocks.append(
            f"<example>\n<input>{ex['input']}</input>\n"
            f"<output>{ex['output']}</output>\n</example>"
        )
    return "\n".join(blocks)
 
def build_prompt(instruction: str, examples: list[dict], query: str) -> str:
    # Keep examples and query in separate sections so the model can see the boundary.
    return (
        f"{instruction}\n\n"
        f"<examples>\n{render_examples(examples)}\n</examples>\n\n"
        f"<input>{query}</input>\n"
    )

For retrieval-selected examples, rank a pool by embedding similarity to the query, take the top few, and deduplicate near-identical cases. Re-ranking by similarity alone can produce a set that is too homogeneous; add one deliberately dissimilar boundary case to keep the decision boundary visible.

Trade-offs

ApproachGood forCostFails when
Zero-shotSimple, well-described tasksLowest token costThe format or boundary is hard to describe
Fixed few-shot setStable format, consistent categoriesInput tokens on every callInputs vary widely in style or domain
Retrieval-selected examplesHeterogeneous corporaSelection latency and infrastructureThe example pool is small or unlabelled
Fine-tuningHigh volume, stable taskTraining data and pipelineRequirements change monthly
Trade-off mapInteractive

Good for

Simple, well-described tasks

Cost

Lowest token cost

Fails when

The format or boundary is hard to describe

  1. The format or boundary is hard to describe
  2. Inputs vary widely in style or domain
  3. The example pool is small or unlabelled
  4. Requirements change monthly

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Zero-shot is good for Simple, well-described tasks, costs Lowest token cost, and fails when The format or boundary is hard to describe. Fixed few-shot set is good for Stable format, consistent categories, costs Input tokens on every call, and fails when Inputs vary widely in style or domain. Retrieval-selected examples is good for Heterogeneous corpora, costs Selection latency and infrastructure, and fails when The example pool is small or unlabelled. Fine-tuning is good for High volume, stable task, costs Training data and pipeline, and fails when Requirements change monthly

Failure modes

Majority-label bias. The model predicts the label that dominates the examples. Debug by measuring per-class recall, not just accuracy, and balance the demonstration set.

Format leakage. The model copies an artefact from the examples — a sample name, a date, a citation marker. Debug by diffing outputs against the example text and by replacing real entities in examples with clearly synthetic ones.

Unrepresentative examples. All demonstrations come from one customer segment, and the model fails on the rest. Debug by stratifying the eval set by segment and sampling examples from the same strata.

Eval contamination. The examples include the exact inputs used for evaluation, so scores are inflated. Debug by deduplicating the demonstration pool against the eval set and keeping a held-out slice that never enters a prompt.

Failure modes, one at a timeInteractive

Symptom

The model predicts the label that dominates the examples.

Debug step

Measuring per-class recall, not just accuracy, and balance the demonstration set.

Figure. Each failure with its symptom and the check that would confirm it.

What to notice. A failure without a debug step is a rumour. Pick the symptom you are seeing; the check is the next action.

Text version

Failure modes: Majority-label bias: The model predicts the label that dominates the examples. Debug by measuring per-class recall, not just accuracy, and balance the demonstration set. | Format leakage: The model copies an artefact from the examples — a sample name, a date, a citation marker. Debug by diffing outputs against the example text and by replacing real entities in examples with clearly synthetic ones. | Unrepresentative examples: All demonstrations come from one customer segment, and the model fails on the rest. Debug by stratifying the eval set by segment and sampling examples from the same strata. | Eval contamination: The examples include the exact inputs used for evaluation, so scores are inflated. Debug by deduplicating the demonstration pool against the eval set and keeping a held-out slice that never enters a prompt.

In production

Treat the example set as a versioned asset with an owner, a selection policy and a token budget. Log which examples were used for each request, because "the model changed" often means "the selector returned different examples". Cache the rendered prefix where prefix caching is supported. Review the pool quarterly for stale formats and leaked customer data. If the pool grows past a handful per request, you have a retrieval problem, not a prompting problem.

Common misconceptions

People think: More examples always improve accuracy.

Actually: Accuracy often plateaus or declines as the prompt grows. Extra examples add cost and can introduce conflicting patterns or recency bias.

People think: Few-shot teaches the model new facts.

Actually: It conditions the existing model on a pattern. Facts come from pretraining, retrieval or tools; demonstrations mainly convey format and task framing.

Interview angle

Expect "the classifier works for two labels but never predicts the third". The strong answer checks label balance in the examples, per-class recall, example order and similarity to the failing input, then proposes retrieval-selected examples or a fine-tune if volume justifies it. The weak answer adds more examples of the missing class without measuring. See the prompting bank.

Sources