juniorReportedcurated
The PM says: fine-tune the model on our docs so it knows our product. How do you respond?
30-second answer
- Fine-tune for behaviour, retrieve for knowledge, and prompt first: if the gap is facts about the product, retrieval is the fix because it is fresh, private and citable.
- SFT on documents gives unreliable recall, cannot cite, and goes stale immediately, so it is the wrong tool for a knowledge problem even when the documents are clean.
- Fine-tuning wins for format, tone, a narrow skill such as classification or routing, or cutting a 2,000-token few-shot prompt out of every request.
- The escalation ladder is prompting, then RAG, then LoRA, and I build the evaluation set before any of it so improvement can be demonstrated rather than asserted.
- RAG adds retrieval latency and a serving component that tuning would remove, so the choice is a trade-off between freshness and operational surface, not between good and bad.
Likely follow-ups
Open a question to rehearse the answer.
What eval would you build before deciding?
- Build a golden set from real support questions with the correct answer and supporting document, and score the prompted baseline first.
- Separate metrics for answer correctness, citation coverage and abstention, because a model can sound right while citing nothing.
- Slice by question type so a retrieval fix is not credited for gains on questions that never needed retrieval.
When is the honest answer both RAG and fine-tuning?
- Use retrieval for facts and fine-tuning for the behaviour around them, such as citation format, tool-call syntax or routing to the right index.
- Evaluate each independently: fix retrieval first, then tune only the residual behaviour the prompt cannot hold.
- Keep the fine-tune free of volatile facts, or every policy update becomes a retraining job.
Red flags
- Agrees to fine-tune on documents to inject facts
- Has no baseline metric, so improvement cannot be demonstrated
Concepts this tests
Tags
Source: github.com