Contrastive learning-based methods, such as unsup-SimCSE, have achieved
state-of-the-art (SOTA) performances in learning unsupervised sentence
embeddings. However, in previous studies, each embedding used for contrastive
learning only derived from one sentence instance, and we call these embeddings
instance-level embeddings. In other words, each embedding is regarded as a
unique class of its own, whichmay hurt the generalization performance. In this
study, we propose IS-CSE (instance smoothing contrastive sentence embedding) to
smooth the boundaries of embeddings in the feature space. Specifically, we
retrieve embeddings from a dynamic memory buffer according to the semantic
similarity to get a positive embedding group. Then embeddings in the group are
aggregated by a self-attention operation to produce a smoothed instance
embedding for further analysis. We evaluate our method on standard semantic
text similarity (STS) tasks and achieve an average of 78.30%, 79.47%, 77.73%,
and 79.42% Spearman's correlation on the base of BERT-base, BERT-large,
RoBERTa-base, and RoBERTa-large respectively, a 2.05%, 1.06%, 1.16% and 0.52%
improvement compared to unsup-SimCSE.

本文提出了 IS-CSE 方法用于平滑特征空间中的嵌入，实现句子嵌入的无监督学习，进而在标准的语义文本相似度（STS）任务中取得更好的结果。

无监督句子嵌入的实例平滑对比学习

Instance Smoothed Contrastive Learning for Unsupervised Sentence  Embedding

Sentence embedding methods using natural language inference (NLI) datasets
have been successfully applied to various tasks. However, these methods are
only available for limited languages due to relying heavily on the large NLI
datasets. In this paper, we propose DefSent, a sentence embedding method that
uses definition sentences from a word dictionary, which performs comparably on
unsupervised semantics textual similarity (STS) tasks and slightly better on
SentEval tasks than conventional methods. Since dictionaries are available for
many languages, DefSent is more broadly applicable than methods using NLI
datasets without constructing additional datasets. We demonstrate that DefSent
performs comparably on unsupervised semantics textual similarity (STS) tasks
and slightly better on SentEval tasks to the methods using large NLI datasets.
Our code is publicly available at this https URL .

该研究提出了一种基于单词词典中定义句子的句子嵌入方法 DefSent，其在文本相似性和情感分析任务上相较于常规方法表现更好，以较小的代价拥有与大型 NLI 数据库相似的性能。

DefSent: 使用定义句子生成句子嵌入

DefSent: Sentence Embeddings using Definition Sentences

Recently, BERT realized significant progress for sentence matching via
word-level cross sentence attention. However, the performance significantly
drops when using siamese BERT-networks to derive two sentence embeddings, which
fall short in capturing the global semantic since the word-level attention
between two sentences is absent. In this paper, we propose a Dual-view
distilled BERT~(DvBERT) for sentence matching with sentence embeddings. Our
method deals with a sentence pair from two distinct views, i.e., Siamese View
and Interaction View. Siamese View is the backbone where we generate sentence
embeddings. Interaction View integrates the cross sentence interaction as
multiple teachers to boost the representation ability of sentence embeddings.
Experiments on six STS tasks show that our method outperforms the
state-of-the-art sentence embedding methods significantly.

本文提出了一种基于 Dual-view distilled BERT~(DvBERT) 的句子匹配方法，该方法能够捕捉多个角度的句子信息，进而提升表征能力，相较于 state-of-the-art 方法，在六个 STS 任务上达到了更好的效果。