Unsupervised sentence embeddings task aims to convert sentences to semantic
vector representations. Most previous works directly use the sentence
representations derived from pretrained language models. However, due to the
token bias in pretrained language models, the models can not capture the
fine-grained semantics in sentences, which leads to poor predictions. To
address this issue, we propose a novel Self-Adaptive Reconstruction Contrastive
Sentence Embeddings (SARCSE) framework, which reconstructs all tokens in
sentences with an AutoEncoder to help the model to preserve more fine-grained
semantics during tokens aggregating. In addition, we proposed a self-adaptive
reconstruction loss to alleviate the token bias towards frequency. Experimental
results show that SARCSE gains significant improvements compared with the
strong baseline SimCSE on the 7 STS tasks.

我们提出了一种自适应重建对比句子嵌入（SARCSE）框架，通过使用自动编码器对句子中的所有单词进行重建，帮助模型在聚合单词时保留更多细粒度的语义。实验结果表明，与强基准 SimCSE 相比，SARCSE 在 7 个 STS 任务中取得了显着改进。