Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 2 of 28

Foundation models

0 of 28 complete

Why it matters

Before foundation models, a team built a model per task with its own labels, training run and maintenance. That made each task expensive and rare tasks uneconomic. A foundation model shifts the cost: pretraining is enormous and centralised, while adaptation is cheap and local. The engineering problem moves from training to specification, evaluation and lifecycle management of a component you did not build and cannot retrain.

Mental model

A foundation model is an engine sold to many vehicle makers: one design, adapted through the chassis around it. The comparison breaks because the engine's behaviour changes when the supplier retrains it, while a mechanical engine keeps its specification. Your adaptation is real work, but it sits on a design you do not control.

How it works

Mechanism, step by stepInteractive
  1. 1Separate what the base model supplies, such as language and format following, from what your application must supply, such as private facts, permissions and schema.
  2. 2Choose adaptation: prompt and context for behaviour, retrieval for knowledge, fine-tuning only when style or a narrow task justifies a training loop.
  3. 3Freeze an evaluation set before adapting, so gains from prompting are distinguishable from noise.
  4. 4Treat the base model as a versioned dependency and record the exact model id, provider and date in every request log.
  5. 5Plan for deprecation by keeping a second model that passes the same evaluation, even if it carries no traffic.
  6. 6Re-run the evaluation whenever the base model, its tokeniser or its serving stack changes.

Step 1 of 6

Separate what the base model supplies, such as language and format following, from what your application must supply, such as private facts, permissions and schema.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Separate what the base model supplies, such as language and format following, from what y… 2. Choose adaptation: prompt and context for behaviour, retrieval for knowledge, fine-tuning… 3. Freeze an evaluation set before adapting, so gains from prompting are distinguishable fro… 4. Treat the base model as a versioned dependency and record the exact model id, provider an… 5. Plan for deprecation by keeping a second model that passes the same evaluation, even if i… 6. Re-run the evaluation whenever the base model, its tokeniser or its serving stack changes.

  1. Separate what the base model supplies, such as language and format following, from what your application must supply, such as private facts, permissions and schema.
  2. Choose adaptation: prompt and context for behaviour, retrieval for knowledge, fine-tuning only when style or a narrow task justifies a training loop.
  3. Freeze an evaluation set before adapting, so gains from prompting are distinguishable from noise.
  4. Treat the base model as a versioned dependency and record the exact model id, provider and date in every request log.
  5. Plan for deprecation by keeping a second model that passes the same evaluation, even if it carries no traffic.
  6. Re-run the evaluation whenever the base model, its tokeniser or its serving stack changes.

Trade-offs

ChoiceGood forCostFails when
Prompt-only adaptationFastest to ship, reversibleEvery request pays for instructions and contextThe task needs behaviour the base model never learned
Fine-tuningStable style, shorter promptsTraining data, evaluation and hosting workFacts change faster than you can retrain
RetrievalFresh private knowledgeRetrieval quality becomes the failure surfaceThe task is a skill, not a lookup
One base model for all featuresOne integration, shared evaluationOne regression affects every featureFeatures have very different latency or safety needs
Trade-off mapInteractive

Good for

Fastest to ship, reversible

Cost

Every request pays for instructions and context

Fails when

The task needs behaviour the base model never learned

  1. The task needs behaviour the base model never learned
  2. Facts change faster than you can retrain
  3. The task is a skill, not a lookup
  4. Features have very different latency or safety needs

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Prompt-only adaptation is good for Fastest to ship, reversible, costs Every request pays for instructions and context, and fails when The task needs behaviour the base model never learned. Fine-tuning is good for Stable style, shorter prompts, costs Training data, evaluation and hosting work, and fails when Facts change faster than you can retrain. Retrieval is good for Fresh private knowledge, costs Retrieval quality becomes the failure surface, and fails when The task is a skill, not a lookup. One base model for all features is good for One integration, shared evaluation, costs One regression affects every feature, and fails when Features have very different latency or safety needs

Failure modes

  • Capability assumed, not measured. The team ships a task the base model cannot do at the required accuracy because a demo looked convincing. Debug: run the frozen evaluation set before writing the integration and record the baseline.
  • Silent base-model change. A provider updates the model behind a stable name and outputs shift. Debug: pin a dated model id and diff evaluation scores across the upgrade.
  • Knowledge and skill confused. The team fine-tunes to add facts that change weekly. Debug: test a retrieval-only variant on the same questions, which separates missing knowledge from missing behaviour.

In production

Track evaluation scores per base-model version and keep the previous version reachable until the new one passes. Budget time for re-running the full suite on every deprecation notice.

Common misconceptions

People think: A foundation model is a general product that needs no adaptation.

Actually: It supplies general capability; your application still supplies the task definition, private context and output contract.

People think: Fine-tuning is how you add knowledge.

Actually: Fine-tuning shapes behaviour and format more reliably than it stores facts; changing knowledge usually belongs in retrieval.

Interview angle

Strong answers separate capability, knowledge and behaviour, then say which adaptation handles each and how it is evaluated. They also mention lifecycle: version pinning and a tested fallback. Weak answers treat the model as a fixed library and never mention what happens when the provider changes it.

Sources