Distance Metric Learning (DML) has attracted much attention in image
processing in recent years. This paper analyzes its impact on supervised
fine-tuning language models for Natural Language Processing (NLP)
classification tasks under few-shot learning settings. We investigated several
DML loss functions in training RoBERTa language models on known SentEval
Transfer Tasks datasets. We also analyzed the possibility of using proxy-based
DML losses during model inference.
Our systematic experiments have shown that under few-shot learning settings,
particularly proxy-based DML losses can positively affect the fine-tuning and
inference of a supervised language model. Models tuned with a combination of
CCE (categorical cross-entropy loss) and ProxyAnchor Loss have, on average, the
best performance and outperform models with only CCE by about 3.27 percentage
points -- up to 10.38 percentage points depending on the training dataset.

本论文研究了距离度量学习 (Distance Metric Learning) 对少样本学习下的监督学习语言模型 Fine-tuning 在自然语言处理 (NLP) 分类任务中的影响，并发现使用基于代理的 DML 损失有助于模型的精调和推理，尤其是使用 CCE（分类交叉熵）和 ProxyAnchor Loss 的结合模型，其平均性能优于仅使用 CCE 的模型，提高了 3.27 个百分点 -- 具体取决于训练数据集的不同。