Backward Compatible Training

Backward-compatible training (BCT) aims to update large-scale retrieval systems with improved models without the computationally expensive process of recomputing all existing data embeddings (backfilling). Current research focuses on methods that align new and old model representations, often employing techniques like orthogonal transformations, projection modules, or adversarial learning to maintain compatibility while preserving or improving the new model's performance. This addresses a critical challenge in deploying updated models in applications like image and cross-modal retrieval, enabling faster and more efficient system upgrades with significant cost savings and improved user experience.