We address the task of unsupervised Semantic Textual Similarity (STS) by
ensembling diverse pre-trained sentence encoders into sentence meta-embeddings.
We apply, extend and evaluate different meta-embedding methods from the word
embedding literature at the sentence level, including dimensionality reduction
(Yin and Sch\"utze, 2016), generalized Canonical Correlation Analysis (Rastogi
et al., 2015) and cross-view auto-encoders (Bollegala and Bao, 2018). Our
sentence meta-embeddings set a new unsupervised State of The Art (SoTA) on the
STS Benchmark and on the STS12-STS16 datasets, with gains of between 3.7% and
6.4% Pearson's r over single-source systems.

本文介绍了如何通过将不同的预训练句子编码器组合成句子元嵌入来解决无监督的语义文本相似性（STS）任务，我们在句子级别应用、扩展和评估了来自单词嵌入文献的不同元嵌入方法，并在 STS 基准测试和 STS12-STS16 数据集上设置了新的无监督状态，平均提高了 3.7% 至 6.4% Pearson 的 r 值。

无监督语义文本相似性的句子元嵌入

Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity

Similarity measures based purely on word embeddings are comfortably competing
with much more sophisticated deep learning and expert-engineered systems on
unsupervised semantic textual similarity (STS) tasks. In contrast to commonly
used geometric approaches, we treat a single word embedding as e.g. 300
observations from a scalar random variable. Using this paradigm, we first
illustrate that similarities derived from elementary pooling operations and
classic correlation coefficients yield excellent results on standard STS
benchmarks, outperforming many recently proposed methods while being much
faster and trivial to implement. Next, we demonstrate how to avoid pooling
operations altogether and compare sets of word embeddings directly via
correlation operators between reproducing kernel Hilbert spaces. Just like
cosine similarity is used to compare individual word vectors, we introduce a
novel application of the centered kernel alignment (CKA) as a natural
generalisation of squared cosine similarity for sets of word vectors. Likewise,
CKA is very easy to implement and enjoys very strong empirical results.

研究表明，基于单词嵌入的相似性度量方法在无监督语义文本相似性（STS）任务中正在与更复杂的深度学习和专家设计的系统轻松竞争。通过将一个单词嵌入视为标量随机变量的例如 300 个观测值，我们避免了传统的几何方法，使用于基本汇集运算和经典相关系数得到的相似性效果很好，胜过许多最近的方法，同时速度更快且实现容易。此外，研究认为通过重新生成核希尔伯特空间之间的相关算子，可以避免汇集运算并直接比较词嵌入集。正如余弦相似性用于比较单个单词向量一样，我们介绍了中心核对齐（CKA）的一种新颖应用，作为平方余弦相似性的集合的自然推广。同样，CKA 非常易于实现，并享有非常强的实证结果。