Modern retrieval systems often struggle with upgrading to new and more
powerful models due to the incompatibility of embeddings between the old and
new models. This necessitates a costly process known as backfilling, which
involves re-computing the embeddings for a large number of data samples. In
vision, Backward-compatible Training (BT) has been proposed to ensure that the
new model aligns with the old model's embeddings. This paper extends the
concept of vision-only BT to the field of cross-modal retrieval, marking the
first attempt to address Cross-modal BT (XBT). Our goal is to achieve
backward-compatibility between Vision-Language Pretraining (VLP) models, such
as CLIP, for the cross-modal retrieval task. To address XBT challenges, we
propose an efficient solution: a projection module that maps the new model's
embeddings to those of the old model. This module, pretrained solely with text
data, significantly reduces the number of image-text pairs required for XBT
learning, and, once it is pretrained, it avoids using the old model during
training. Furthermore, we utilize parameter-efficient training strategies that
improve efficiency and preserve the off-the-shelf new model's knowledge by
avoiding any modifications. Experimental results on cross-modal retrieval
datasets demonstrate the effectiveness of XBT and its potential to enable
backfill-free upgrades when a new VLP model emerges.

本文提出了一种名为 “Cross-modal BT (XBT)” 的方法，通过引入一个经过预训练的投影模块来解决视觉 - 语言预训练模型（如 CLIP）的向后兼容问题，有效降低了跨模态训练所需的图像 - 文本对数量并提高了效率，实验结果显示 XBT 的有效性及其在新 VLP 模型出现时实现无需回填的升级的潜力。

面向视觉 - 语言模型的跨模态向后兼容表示学习

Towards Cross-modal Backward-compatible Representation Learning for  Vision-Language Models

The exponential growth of data, alongside advancements in model structures
and loss functions, has necessitated the enhancement of image retrieval systems
through the utilization of new models with superior feature embeddings.
However, the expensive process of updating the old retrieval database by
replacing embeddings poses a challenge. As a solution, backward-compatible
training can be employed to avoid the necessity of updating old retrieval
datasets. While previous methods achieved backward compatibility by aligning
prototypes of the old model, they often overlooked the distribution of the old
features, thus limiting their effectiveness when the old model's low quality
leads to a weakly discriminative feature distribution. On the other hand,
instance-based methods like L2 regression take into account the distribution of
old features but impose strong constraints on the performance of the new model
itself. In this paper, we propose MixBCT, a simple yet highly effective
backward-compatible training method that serves as a unified framework for old
models of varying qualities. Specifically, we summarize four constraints that
are essential for ensuring backward compatibility in an ideal scenario, and we
construct a single loss function to facilitate backward-compatible training.
Our approach adaptively adjusts the constraint domain for new features based on
the distribution of the old embeddings. We conducted extensive experiments on
the large-scale face recognition datasets MS1Mv3 and IJB-C to verify the
effectiveness of our method. The experimental results clearly demonstrate its
superiority over previous methods. Code is available at
this https URL

提出了 MixBCT，一种简单而高效的反向兼容训练方法，作为适用于不同质量旧模型的统一框架，通过构建单一损失函数实现了反向兼容训练。

自适应向后兼容训练的 MixBCT

MixBCT: Towards Self-Adapting Backward-Compatible Training

Image retrieval plays an important role in the Internet world. Usually, the
core parts of mainstream visual retrieval systems include an online service of
the embedding model and a large-scale vector database. For traditional model
upgrades, the old model will not be replaced by the new one until the
embeddings of all the images in the database are re-computed by the new model,
which takes days or weeks for a large amount of data. Recently,
backward-compatible training (BCT) enables the new model to be immediately
deployed online by making the new embeddings directly comparable to the old
ones. For BCT, improving the compatibility of two models with less negative
impact on retrieval performance is the key challenge. In this paper, we
introduce AdvBCT, an Adversarial Backward-Compatible Training method with an
elastic boundary constraint that takes both compatibility and discrimination
into consideration. We first employ adversarial learning to minimize the
distribution disparity between embeddings of the new model and the old model.
Meanwhile, we add an elastic boundary constraint during training to improve
compatibility and discrimination efficiently. Extensive experiments on GLDv2,
Revisited Oxford (ROxford), and Revisited Paris (RParis) demonstrate that our
method outperforms other BCT methods on both compatibility and discrimination.
The implementation of AdvBCT will be publicly available at
this https URL

介绍了在视觉检索系统中，从传统模型更新到新模型需要重新计算所有图片嵌入，所以引入兼容性训练（BCT）的方法，并提出了 Adversarial Backward-Compatible Training（AdvBCT）方法进行对比实验，表明其在兼容性和辨别性方面均优于其他 BCT 方法。