The impressive performance of GPT-3 using natural language prompts and in-context learning has inspired work on better fine-tuning of moderately-sized models under this paradigm. Following this line of work, we present a contrastive learning framework that clusters inputs from the same class for better generality of models trained with only limited examples. Specifically, we propose a supervised contrastive framework that clusters inputs from the same class under different augmented "views" and repel the ones from different classes. We create different "views" of an example by appending it with different language prompts and contextual demonstrations. Combining a contrastive loss with the standard masked language modeling (MLM) loss in prompt-based few-shot learners, the experimental results show that our method can improve over the state-of-the-art methods in a diverse set of 15 language tasks. Our framework makes minimal assumptions on the task or the base model, and can be applied to many recent methods with little modification. The code will be made available at: https://github.com/yiren-jian/LM-SupCon.

本文提出一种基于对比学习的框架，使用不同的增强“视图”将同一类别的输入聚类，远离来自不同类别的输入，将对比损失与标准的掩码语言建模（MLM）损失相结合，并应用于基于提示的少样本学习者，实验结果表明，我们的方法在15种不同的语言任务中表现优于现有的先进方法。

基于提示的少样本语言学习的对比学习