Text analysis in the social sciences often involves using specialized
dictionaries to reason with abstract concepts, such as perceptions about the
economy or abuse on social media. These dictionaries allow researchers to
impart domain knowledge and note subtle usages of words relating to a
concept(s) of interest. We introduce the dictionary-assisted supervised
contrastive learning (DASCL) objective, allowing researchers to leverage
specialized dictionaries when fine-tuning pretrained language models. The text
is first keyword simplified: a common, fixed token replaces any word in the
corpus that appears in the dictionary(ies) relevant to the concept of interest.
During fine-tuning, a supervised contrastive objective draws closer the
embeddings of the original and keyword-simplified texts of the same class while
pushing further apart the embeddings of different classes. The
keyword-simplified texts of the same class are more textually similar than
their original text counterparts, which additionally draws the embeddings of
the same class closer together. Combining DASCL and cross-entropy improves
classification performance metrics in few-shot learning settings and social
science applications compared to using cross-entropy alone and alternative
contrastive and data augmentation methods.

本文介绍了一种使用字典辅助的有监督对比学习模型，使研究人员能够在微调预训练语言模型时利用专门的字典。这种方法可以在少量样本情况下提高分类性能指标和社会科学应用。