We present a novel retrofitting method to induce emotion aspects into
pre-trained language models (PLMs) such as BERT and RoBERTa. Our method updates
pre-trained network weights using contrastive learning so that the text
fragments exhibiting similar emotions are encoded nearby in the representation
space, and the fragments with different emotion content are pushed apart. While
doing so, it also ensures that the linguistic knowledge already present in PLMs
is not inadvertently perturbed. The language models retrofitted by our method,
i.e., BERTEmo and RoBERTaEmo, produce emotion-aware text representations, as
evaluated through different clustering and retrieval metrics. For the
downstream tasks on sentiment analysis and sarcasm detection, they perform
better than their pre-trained counterparts (about 1% improvement in F1-score)
and other existing approaches. Additionally, a more significant boost in
performance is observed for the retrofitted models over pre-trained ones in
few-shot learning setting.

我们提出了一种新颖的改进方法，用于将情绪方面引入预训练语言模型（如 BERT 和 RoBERTa）。通过对比学习更新预训练网络权重，使表现出类似情绪的文本片段在表示空间中靠近编码，而具有不同情绪内容的片段则被推开。我们的方法对 PLMs 进行了改进，即 BERTEmo 和 RoBERTaEmo，生成了情绪感知的文本表示，通过不同的聚类和检索指标进行评估。在情感分析和讽刺检测的下游任务中，它们比预训练模型表现得更好（F1 分数提高约 1%），且在少样本学习中观察到了更显著的性能提升。