An embedding-model migration is a data and retrieval migration, not a configuration flip. Old and new document embeddings must coexist while you backfill, build a metric-compatible index, embed queries in the new space, compare quality and latency, and cut traffic over gradually. Keep rollback until new writes, reads, deletions, and recovery are proven.
Never compare an old query vector against a new document index.
Slice by tenant, language, source type, and sensitivity. Overall 99% can hide a completely missing tenant.
Generate both query embeddings only for approved sampled traffic. Run the new retrieval without changing the user response. Log result IDs, distances, latency, and model version under privacy policy; do not log raw sensitive queries by default. Compare with labels or outcome proxies, not top-k overlap alone.
Test new ingestion, updates during backfill, deletions, access revocation, failed items, ANN plans, exact recall, failover, and rollback. Freeze the evaluation set before final comparison. Confirm every serving query selects model and index as one configuration unit.
Create a zero-downtime migration runbook from embedding contract A to B [details]. Include schema, dual-write/backfill races, separate indexes, slice completeness, shadow evaluation, traffic gates, rollback triggers, deletion propagation, capacity, and final cleanup.
Verification contract: Rehearse cutover and rollback on a production-shaped snapshot; prove that every served result records one consistent model contract.