Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 1 of 22

Adaptation decision tree

0 of 22 complete

Why it matters

Teams reach for fine-tuning because it sounds like the thorough option, then spend weeks teaching a model a format that a schema and two examples would have enforced. A decision tree makes the cheapest sufficient intervention the default. It asks what the model is failing at, because knowledge gaps, output-shape gaps and behaviour gaps have different remedies, and only some of them are training problems at all.

Mental model

Choosing an adaptation is triage in an emergency room: treat the condition that is actually dangerous, not the one that is loudest. The analogy breaks because a model can be treated twice; you can add retrieval and still tune later, and the order matters less than the cost of undoing a bad tune.

How it works

Four rungs from prompt to trainingInteractive
11 nodes · 5 branches
rungstop pointnotequestion

Step 1 of 7

Classify the failure

Write three examples and label each as missing knowledge, wrong shape, wrong tone or weak reasoning. The label is what makes a rung applicable; without it, any rung looks plausible.

question · current

Failure, written with 3 examples

Start from the observed failure, not from the tool. Three examples are enough to classify it, and the class chooses the first rung that can reach it.

Root of the hierarchy

Also in focus

  • Rung 1 · Prompt + schema
  • Rung 2 · Retrieval
  • Rung 3 · Adapter (LoRA)
  • Rung 4 · Full fine-tune
Step 1/7 · Classify the failure

Figure. Four rungs in cost order, each with the condition that makes it the right first attempt and the stop that ends the descent. A fifth branch closes the ladder with train-from-scratch, outside the page's four rungs; prompt and schema come first in every case.

What to notice. The edge label is the decision: escalation is justified by the failure class, not by which team owns accelerators. Every rung is a stop point, so the cheapest sufficient fix is the default and each step up buys capacity with data, compute and maintenance. Train from scratch is drawn only to close the ladder — it sits outside the page's four rungs, and nothing here recommends it. Costs are relative, not measured.

Text version

A cost ladder drawn as a decision tree. The root is a failure written with three examples, labelled as missing knowledge, wrong shape, wrong tone or weak reasoning. Four rungs hang from it in cost order. Rung 1, prompt and schema, applies first in every case, with a stop if the twenty failures now pass. Rung 2, retrieval, applies when knowledge changes or must be cited, with a stop when grounding closes the gap. Rung 3, an adapter, applies when the behaviour is stable and survives prompting, with a stop when the held-out set and general slice hold. Rung 4, full fine-tuning, applies to a new tokenizer, a new modality or a large shift, with a stop only when the cheaper rung fails. A fifth branch, train from scratch, covers the case where no base model hosts the behaviour and is labelled as outside the tree. A final branch records the decision and its reversal plan beside the model version.

  1. Write down the observed failure with three examples, and label each as missing knowledge, wrong shape, wrong tone or weak reasoning.
  2. Try the prompt first, because a schema, two demonstrations and an explicit refusal rule fix most shape and tone complaints without training.
  3. If the gap is knowledge that changes or must be cited, add retrieval and measure whether grounding closes it.
  4. If the behaviour is stable and survives prompting, compare a small adapter against a hosted fine-tune on the same held-out set.
  5. Escalate to full fine-tuning only when the adapter cannot represent the change, for example a new tokenizer, a new modality or a large distribution shift.
  6. Record the decision and the evidence beside the model version so the next team does not relitigate it.

Trade-offs

ChoiceGood forCostFails when
Prompt and schema onlyFast, reversible, no data pipelineCeiling on style and format consistencyThe task needs behaviour the base model never shows
RetrievalFresh, citable knowledge without trainingRetrieval latency and index maintenanceThe failure is tone or format rather than facts
Adapter tuningCheap iteration, many variants per baseData curation and an eval harnessThe change needs new vocabulary or a new modality
Full fine-tuningMaximum capacity for distribution shiftCost, checkpoints and regression riskA cheaper method already passes the held-out set
Trade-off mapInteractive

Good for

Fast, reversible, no data pipeline

Cost

Ceiling on style and format consistency

Fails when

The task needs behaviour the base model never shows

  1. The task needs behaviour the base model never shows
  2. The failure is tone or format rather than facts
  3. The change needs new vocabulary or a new modality
  4. A cheaper method already passes the held-out set

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Prompt and schema only is good for Fast, reversible, no data pipeline, costs Ceiling on style and format consistency, and fails when The task needs behaviour the base model never shows. Retrieval is good for Fresh, citable knowledge without training, costs Retrieval latency and index maintenance, and fails when The failure is tone or format rather than facts. Adapter tuning is good for Cheap iteration, many variants per base, costs Data curation and an eval harness, and fails when The change needs new vocabulary or a new modality. Full fine-tuning is good for Maximum capacity for distribution shift, costs Cost, checkpoints and regression risk, and fails when A cheaper method already passes the held-out set

Failure modes

  • Skipping the cheap rung. A prompt fix was available but a training run was funded first. Debug: run the prompt and retrieval baseline on the same twenty failures before approving any training budget.
  • Decision by org chart. The team that owns accelerators gets the work regardless of the failure type. Debug: attach a failure label to each request and check whether the chosen remedy matches it.
  • No reversal plan. The tuned model regresses on general traffic and there is no path back to the base model. Debug: keep the base model and prompt version deployable and compare on a general slice before cutover.

In production

Keep the decision record next to the deployed model and revisit it when the base model changes, because an upgrade can make an adapter redundant or harmful. Monitor a general-traffic slice, not only the target task.

Common misconceptions

People think: Fine-tuning is the serious option and prompting is the shortcut.

Actually: Prompting, schemas and retrieval are engineering interventions with their own evaluation; training is the expensive escalation, not the mature default.

People think: If fine-tuning works, it replaces the prompt.

Actually: A tuned model still runs inside a prompt, and the system message, tools and refusal rules usually stay.

Interview angle

Walk the tree out loud: name the failure class first, then the cheapest remedy that could fix it, then the evidence that would justify the next rung. Strong candidates mention what makes an adaptation reversible. Weak answers jump to a training recipe and never say which failure it addresses or how they would know it worked.

Sources