Why it matters
Neural networks do not partition capacity by task. When training data shifts, the weights that produced old behaviour receive no gradient telling them to stay, so they drift toward whatever the new loss rewards. The result is a model that passes the new evaluation and fails on traffic it used to handle. The engineering problem is preserving behaviour you are not measuring.
Mental model
Forgetting is renovating one room and finding the plumbing inside the wall you removed: the change is local in intent and systemic in effect. The analogy breaks because you can inspect the old behaviour directly by running the base model on the same prompts.
How it works
Adapter · Step 1 of 5
Measure first
Score the base on a broad slice before training. Without that reference, a regression is indistinguishable from a change in preference.
Active layers
- Base checkpoint
- General slice
Hover a layer to read what it holds.
Next: Hold the base bands
- Updated
- adapter weights only
- Frozen
- embeddings · attention · FFN
- Risk
- weak adapter
Figure. The same architecture under two update scopes. In the adapter arrangement the three base bands are frozen and only the adapter moves; in the full fine-tune every band is trainable and a narrow target loss reaches all of them. The general slice is measured before and after in both. The switch selects one complete stack at a time; mixing general data is a mitigation named in the steps, not a drawn band.
What to notice. Freezing is a property of the bands, not of the step order: the adapter arrangement cannot forget the base because those weights are never in the optimiser. The full arrangement is the failure case, and its sign is two curves separating, target up and general down. Neither drawing measures the size of the loss; it shows where the update lands and what that makes possible.
Text version
A switch labelled Update scope selects Adapter or Full fine-tune, each a complete vertical stack. Adapter: a base checkpoint feeds a frozen group holding embeddings, attention and feed-forward, and a dashed lane from the base attaches a single trainable adapter per layer; both the frozen forward path and the adapter reach the tuned model, which then reaches a general slice; a second dashed lane carries the base to the same slice as a baseline. Full fine-tune: the same base feeds a trainable group with the same three bands marked updated, whose logits enter a narrow target loss, then an optimiser update; the update writes the tuned model and returns to the group along a dash-dot lane labelled every weight, and a dashed lane anchors the baseline. Five steps per arrangement cover the baseline, the update scope, the update, the narrowing loss, and the two curves.
- Measure the base model on a broad slice before training, so you have a reference for what is being lost.
- Mix a share of general instruction data into the fine-tuning set, with the ratio as an explicit parameter.
- Prefer an adapter when the target change is behavioural, since freezing the base confines drift to the added weights.
- Lower the learning rate and shorten the schedule, because most forgetting happens in the first aggressive updates.
- Evaluate after every checkpoint on both the target and the general slice, and plot the two curves together.
- If forgetting persists, interleave batches from both distributions rather than training sequentially.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Mix general data | Cheap, effective, measurable | Dilutes the target signal | The target distribution is very narrow |
| Adapter only | Base behaviour preserved | Capacity limit on the change | You need a new tokenizer or modality |
| Low learning rate | Less drift | Slower convergence | The task needs a large shift |
| Rehearsal buffer | Directly targets old behaviour | Storage and pipeline complexity | You cannot retain the old data |
Good for
Cheap, effective, measurable
Cost
Dilutes the target signal
Fails when
The target distribution is very narrow
- The target distribution is very narrow
- You need a new tokenizer or modality
- The task needs a large shift
- You cannot retain the old data
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Mix general data is good for Cheap, effective, measurable, costs Dilutes the target signal, and fails when The target distribution is very narrow. Adapter only is good for Base behaviour preserved, costs Capacity limit on the change, and fails when You need a new tokenizer or modality. Low learning rate is good for Less drift, costs Slower convergence, and fails when The task needs a large shift. Rehearsal buffer is good for Directly targets old behaviour, costs Storage and pipeline complexity, and fails when You cannot retain the old data
Failure modes
- Target-only evaluation. The tuned model passes and users report it cannot do the old task. Debug: run the base model's evaluation suite against the tuned model and diff per category.
- Silent refusal drift. The model starts refusing benign requests after safety-heavy tuning. Debug: score a fixed benign prompt set before and after, and compare refusal rates.
- Format lock-in. The model applies the new output format to unrelated requests. Debug: sample general prompts and check whether the tuned template appears unprompted.
In production
Keep a regression suite that runs on every checkpoint, not only at release. Alert on refusal rate and format compliance, which move before aggregate quality does.
Common misconceptions
People think: Forgetting only affects very small models.
Actually: It is a property of narrow sequential updates and appears at every scale, though the symptoms differ.
People think: A lower learning rate solves it.
Actually: It slows drift; only data mixing or parameter isolation addresses the cause.
Interview angle
Forgetting is interference from narrow gradients, so give the mitigations in cost order: data mixing, adapters, schedules. Say how you would detect it with a base-model reference suite. Weak answers name the phenomenon and stop, or claim PEFT eliminates it rather than confining it.
Sources
- French, Catastrophic Forgetting in Connectionist Networks — the original statement of the interference problem.
- Kirkpatrick et al., Overcoming Catastrophic Forgetting in Neural Networks — penalising changes to weights important for old tasks.
- Luo et al., An Empirical Study of Catastrophic Forgetting in Large Language Models — how the effect appears during continual fine-tuning.