Deep Metric Learning (DML) provides a crucial tool for visual similarity and
zero-shot applications by learning generalizing embedding spaces, although
recent work in DML has shown strong performance saturation across training
objectives. However, generalization capacity is known to scale with the
embedding space dimensionality. Unfortunately, high dimensional embeddings also
create higher retrieval cost for downstream applications. To remedy this, we
propose \emph{Simultaneous Similarity-based Self-distillation (S2SD). S2SD
extends DML with knowledge distillation from auxiliary, high-dimensional
embedding and feature spaces to leverage complementary context during training
while retaining test-time cost and with negligible changes to the training
time. Experiments and ablations across different objectives and standard
benchmarks show S2SD offers notable improvements of up to 7% in Recall@1, while
also setting a new state-of-the-art. Code available at
this https URL

本文提出了一种名为 S2SD 的新方法，将基于相似性的自我蒸馏应用于 Deep Metric Learning 模型中，通过辅助高维度嵌入和特征空间的知识蒸馏，在保持测试时间成本不变的情况下，实现推理性能的显著提升。实验结果表明，该方法能够有效地提高 Recall@1 指标，同时也创造了新的最优性能记录。