Contrastive learning has been extensively studied in sentence embedding
learning, which assumes that the embeddings of different views of the same
sentence are closer. The constraint brought by this assumption is weak, and a
good sentence representation should also be able to reconstruct the original
sentence fragments. Therefore, this paper proposes an information-aggregated
contrastive learning framework for learning unsupervised sentence embeddings,
termed InfoCSE. InfoCSE forces the representation of [CLS] positions to
aggregate denser sentence information by introducing an additional Masked
language model task and a well-designed network. We evaluate the proposed
InfoCSE on several benchmark datasets w.r.t the semantic text similarity (STS)
task. Experimental results show that InfoCSE outperforms SimCSE by an average
Spearman correlation of 2.60% on BERT-base, and 1.77% on BERT-large, achieving
state-of-the-art results among unsupervised sentence representation learning
methods. Our code are available at
this https URL.

本文提出了一种信息聚合的对比学习框架 InfoCSE 用于学习无监督的句子嵌入，该框架引入了一个额外的掩蔽语言模型任务和一个经过精心设计的网络来强制 [CLS] 位置的表示聚合更密集的句子信息，并在语义文本相似性（STS）任务上取得了超越 SimCSE 的表现，平均 Spearman 相关性增加了 2.60％（BERT-base）和 1.77％（BERT-large），成为无监督句子表示学习方法中的最新成果。