The concept of unsupervised universal sentence encoders has gained traction
recently, wherein pre-trained models generate effective task-agnostic
fixed-dimensional representations for phrases, sentences and paragraphs. Such
methods are of varying complexity, from simple weighted-averages of word
vectors to complex language-models based on bidirectional transformers. In this
work we propose a novel technique to generate sentence-embeddings in an
unsupervised fashion by projecting the sentences onto a fixed-dimensional
manifold with the objective of preserving local neighbourhoods in the original
space. To delineate such neighbourhoods we experiment with several set-distance
metrics, including the recently proposed Word Mover's distance, while the
fixed-dimensional projection is achieved by employing a scalable and efficient
manifold approximation method rooted in topological data analysis. We test our
approach, which we term EMAP or Embeddings by Manifold Approximation and
Projection, on six publicly available text-classification datasets of varying
size and complexity. Empirical results show that our method consistently
performs similar to or better than several alternative state-of-the-art
approaches.

该研究提出了一种名为 EMAP 的新技术，通过将句子投射到一个固定维度的流形上，以保留原始空间中的局部邻域来无监督地生成句子嵌入，可以用于文本分类，实验证明该方法性能优于其他最先进方法。

通过流形逼近和投影实现的无监督句子嵌入

Unsupervised Sentence-embeddings by Manifold Approximation and  Projection

Despite the fast developmental pace of new sentence embedding methods, it is
still challenging to find comprehensive evaluations of these different
techniques. In the past years, we saw significant improvements in the field of
sentence embeddings and especially towards the development of universal
sentence encoders that could provide inductive transfer to a wide variety of
downstream tasks. In this work, we perform a comprehensive evaluation of recent
methods using a wide variety of downstream and linguistic feature probing
tasks. We show that a simple approach using bag-of-words with a recently
introduced language model for deep context-dependent word embeddings proved to
yield better results in many tasks when compared to sentence encoders trained
on entailment datasets. We also show, however, that we are still far away from
a universal encoder that can perform consistently across several downstream
tasks.

本文对最新的句子嵌入方法进行了全面评估，通过使用多样的下游和语言特征探测任务，表明与在蕴涵数据集上训练的句子编码器相比，使用具有深度上下文相关性单词嵌入的词袋模型可以在许多任务中产生更好的结果，但我们远未达成一个可以在多个下游任务中持续表现的通用编码器。