Why it matters
Before foundation models, a team built a model per task with its own labels, training run and maintenance. That made each task expensive and rare tasks uneconomic. A foundation model shifts the cost: pretraining is enormous and centralised, while adaptation is cheap and local. The engineering problem moves from training to specification, evaluation and lifecycle management of a component you did not build and cannot retrain.
Mental model
A foundation model is an engine sold to many vehicle makers: one design, adapted through the chassis around it. The comparison breaks because the engine's behaviour changes when the supplier retrains it, while a mechanical engine keeps its specification. Your adaptation is real work, but it sits on a design you do not control.
How it works
- 1Separate what the base model supplies, such as language and format following, from what your application must supply, such as private facts, permissions and schema.
- 2Choose adaptation: prompt and context for behaviour, retrieval for knowledge, fine-tuning only when style or a narrow task justifies a training loop.
- 3Freeze an evaluation set before adapting, so gains from prompting are distinguishable from noise.
- 4Treat the base model as a versioned dependency and record the exact model id, provider and date in every request log.
- 5Plan for deprecation by keeping a second model that passes the same evaluation, even if it carries no traffic.
- 6Re-run the evaluation whenever the base model, its tokeniser or its serving stack changes.
Step 1 of 6
Separate what the base model supplies, such as language and format following, from what your application must supply, such as private facts, permissions and schema.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Separate what the base model supplies, such as language and format following, from what y… 2. Choose adaptation: prompt and context for behaviour, retrieval for knowledge, fine-tuning… 3. Freeze an evaluation set before adapting, so gains from prompting are distinguishable fro… 4. Treat the base model as a versioned dependency and record the exact model id, provider an… 5. Plan for deprecation by keeping a second model that passes the same evaluation, even if i… 6. Re-run the evaluation whenever the base model, its tokeniser or its serving stack changes.
- Separate what the base model supplies, such as language and format following, from what your application must supply, such as private facts, permissions and schema.
- Choose adaptation: prompt and context for behaviour, retrieval for knowledge, fine-tuning only when style or a narrow task justifies a training loop.
- Freeze an evaluation set before adapting, so gains from prompting are distinguishable from noise.
- Treat the base model as a versioned dependency and record the exact model id, provider and date in every request log.
- Plan for deprecation by keeping a second model that passes the same evaluation, even if it carries no traffic.
- Re-run the evaluation whenever the base model, its tokeniser or its serving stack changes.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Prompt-only adaptation | Fastest to ship, reversible | Every request pays for instructions and context | The task needs behaviour the base model never learned |
| Fine-tuning | Stable style, shorter prompts | Training data, evaluation and hosting work | Facts change faster than you can retrain |
| Retrieval | Fresh private knowledge | Retrieval quality becomes the failure surface | The task is a skill, not a lookup |
| One base model for all features | One integration, shared evaluation | One regression affects every feature | Features have very different latency or safety needs |
Good for
Fastest to ship, reversible
Cost
Every request pays for instructions and context
Fails when
The task needs behaviour the base model never learned
- The task needs behaviour the base model never learned
- Facts change faster than you can retrain
- The task is a skill, not a lookup
- Features have very different latency or safety needs
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Prompt-only adaptation is good for Fastest to ship, reversible, costs Every request pays for instructions and context, and fails when The task needs behaviour the base model never learned. Fine-tuning is good for Stable style, shorter prompts, costs Training data, evaluation and hosting work, and fails when Facts change faster than you can retrain. Retrieval is good for Fresh private knowledge, costs Retrieval quality becomes the failure surface, and fails when The task is a skill, not a lookup. One base model for all features is good for One integration, shared evaluation, costs One regression affects every feature, and fails when Features have very different latency or safety needs
Failure modes
- Capability assumed, not measured. The team ships a task the base model cannot do at the required accuracy because a demo looked convincing. Debug: run the frozen evaluation set before writing the integration and record the baseline.
- Silent base-model change. A provider updates the model behind a stable name and outputs shift. Debug: pin a dated model id and diff evaluation scores across the upgrade.
- Knowledge and skill confused. The team fine-tunes to add facts that change weekly. Debug: test a retrieval-only variant on the same questions, which separates missing knowledge from missing behaviour.
In production
Track evaluation scores per base-model version and keep the previous version reachable until the new one passes. Budget time for re-running the full suite on every deprecation notice.
Common misconceptions
People think: A foundation model is a general product that needs no adaptation.
Actually: It supplies general capability; your application still supplies the task definition, private context and output contract.
People think: Fine-tuning is how you add knowledge.
Actually: Fine-tuning shapes behaviour and format more reliably than it stores facts; changing knowledge usually belongs in retrieval.
Interview angle
Strong answers separate capability, knowledge and behaviour, then say which adaptation handles each and how it is evaluated. They also mention lifecycle: version pinning and a tested fallback. Weak answers treat the model as a fixed library and never mention what happens when the provider changes it.
Sources
- Bommasani et al., On the Opportunities and Risks of Foundation Models — the definition, emergence and risks of the category.
- Ouyang et al., Training language models to follow instructions — instruction tuning as the adaptation that makes a base model usable.
- Kaplan et al., Scaling Laws for Neural Language Models — why capability is a function of pretraining scale.