seniorReportedcurated
Why does swapping the embedding model force a full re-index, and how do you roll that out without downtime?
30-second answer
- Vectors from two models occupy unrelated coordinate spaces, so a query embedded with the new model has no meaningful distance to old document vectors and there is no partial migration.
- The rollout is a second collection built with the new model, a background backfill while the old one serves, dual writes for changes, a golden-set comparison, then an alias flip with the old collection kept for rollback.
- The cost is double storage and double embedding spend for the duration, and deletes and updates must reach both collections or the new index silently diverges.
- Stamp every chunk with model name and version from day one, because a collection with two model versions produces an unexplained recall drop rather than an error.
- This does not avoid the bill: the migration is a deployment with a validation gate and a rollback path, not a script that rewrites vectors in place.
Likely follow-ups
Open a question to rehearse the answer.
How do you keep deletes and updates consistent while the backfill is still running?
- Write through a single ingestion path that applies the same upsert and delete to both collections, keyed by a deterministic chunk id, so replay is idempotent.
- Track a per-document version or updated_at watermark and re-copy anything that changed after its backfill page was read.
- Reconcile document ids and counts between source, old index and new index before the flip, and treat a non-zero miss rate as a blocker.
What rollback trigger would make you abort the migration, and how do you set it from evidence?
- Abort if recall@10 or nDCG@10 on the frozen golden set falls outside the noise band measured on the old collection, or if the empty-result rate rises at all.
- Measure the band by re-running the old model on the same set and against a rebuilt index several times, then set the trigger above that spread and below the regression you would ship.
- Add an operational trigger for backfill lag or dual-write failure, because those corrupt the comparison even when offline scores look fine.
How do you handle a vendor deprecating the old model on 30 days' notice mid-migration?
- Freeze a copy of the old model or its outputs where the licence allows, so rollback remains possible after the vendor deadline.
- Compress the plan to the critical path: backfill first, validate on the golden set, and accept a shorter dual-write window rather than skipping validation.
- Negotiate an extension in parallel and record the deprecation date in the index metadata, since the next migration will hit the same constraint.
Red flags
- mixes vectors from two models
- no rollback plan
Concepts this tests
Tags
Source: www.mockingly.ai
More in Embeddings & retrieval
- Where do embeddings systematically fail as a retrieval mechanism, and which failures can you fix cheaply?
- You must choose an embedding model for a 900k-chunk support corpus. Walk me through the bake-off and the constraints that don't appear on a leaderboard.
- You have 25M chunks at 1,536 dimensions. Size the index and tell me how you make it fit — then what changes at 10x.