Despite the fast developmental pace of new sentence embedding methods, it is
still challenging to find comprehensive evaluations of these different
techniques. In the past years, we saw significant improvements in the field of
sentence embeddings and especially towards the development of universal
sentence encoders that could provide inductive transfer to a wide variety of
downstream tasks. In this work, we perform a comprehensive evaluation of recent
methods using a wide variety of downstream and linguistic feature probing
tasks. We show that a simple approach using bag-of-words with a recently
introduced language model for deep context-dependent word embeddings proved to
yield better results in many tasks when compared to sentence encoders trained
on entailment datasets. We also show, however, that we are still far away from
a universal encoder that can perform consistently across several downstream
tasks.

本文对最新的句子嵌入方法进行了全面评估，通过使用多样的下游和语言特征探测任务，表明与在蕴涵数据集上训练的句子编码器相比，使用具有深度上下文相关性单词嵌入的词袋模型可以在许多任务中产生更好的结果，但我们远未达成一个可以在多个下游任务中持续表现的通用编码器。