Deep Metric Learning (DML) has long attracted the attention of the machine
learning community as a key objective. Existing solutions concentrate on
fine-tuning the pre-trained models on conventional image datasets. As a result
of the success of recent pre-trained models trained from larger-scale datasets,
it is challenging to adapt the model to the DML tasks in the local data domain
while retaining the previously gained knowledge. In this paper, we investigate
parameter-efficient methods for fine-tuning the pre-trained model for DML
tasks. In particular, we propose a novel and effective framework based on
learning Visual Prompts (VPT) in the pre-trained Vision Transformers (ViT).
Based on the conventional proxy-based DML paradigm, we augment the proxy by
incorporating the semantic information from the input image and the ViT, in
which we optimize the visual prompts for each class. We demonstrate that our
new approximations with semantic information are superior to representative
capabilities, thereby improving metric learning performance. We conduct
extensive experiments to demonstrate that our proposed framework is effective
and efficient by evaluating popular DML benchmarks. In particular, we
demonstrate that our fine-tuning method achieves comparable or even better
performance than recent state-of-the-art full fine-tuning works of DML while
tuning only a small percentage of total parameters.

本研究探讨了用于深度度量学习任务的预训练模型的参数有效微调方法，通过学习视觉提示在预训练的视觉转换器中来增强传统的基于代理的深度度量学习模型的代理，并证明了该方法在度量学习性能上超越了代表性能力的，只微调了一小部分参数的新近工作。