Contrastive learning (CL) has achieved astonishing progress in computer
vision, speech, and natural language processing fields recently with
self-supervised learning. However, CL approach to the supervised setting is not
fully explored, especially for the natural language understanding
classification task. Intuitively, the class label itself has the intrinsic
ability to perform hard positive/negative mining, which is crucial for CL.
Motivated by this, we propose a novel label anchored contrastive learning
approach (denoted as LaCon) for language understanding. Specifically, three
contrastive objectives are devised, including a multi-head instance-centered
contrastive loss (ICL), a label-centered contrastive loss (LCL), and a label
embedding regularizer (LER). Our approach does not require any specialized
network architecture or any extra data augmentation, thus it can be easily
plugged into existing powerful pre-trained language models. Compared to the
state-of-the-art baselines, LaCon obtains up to 4.1% improvement on the popular
datasets of GLUE and CLUE benchmarks. Besides, LaCon also demonstrates
significant advantages under the few-shot and data imbalance settings, which
obtains up to 9.4% improvement on the FewGLUE and FewCLUE benchmarking tasks.

本文提出一种基于标签锚定的对比学习方法（LaCon），用于自然语言理解分类任务中，不需要专门的网络架构或任何额外的数据增强，可轻松插入现有的强大预训练语言模型，相比于现有最先进技术，LaCon 在 GLUE 和 CLUE 基准测试数据集上有高达 4.1% 的提升，同时在少量样本和数据失衡的情况下也展现出显著的优势，获取了 FewGLUE 和 FewCLUE 基准测试任务高达 9.4% 的提升。