Probabilistic word embeddings have shown effectiveness in capturing notions
of generality and entailment, but there is very little work on doing the
analogous type of investigation for sentences. In this paper we define
probabilistic models that produce distributions for sentences. Our
best-performing model treats each word as a linear transformation operator
applied to a multivariate Gaussian distribution. We train our models on
paraphrases and demonstrate that they naturally capture sentence specificity.
While our proposed model achieves the best performance overall, we also show
that specificity is represented by simpler architectures via the norm of the
sentence vectors. Qualitative analysis shows that our probabilistic model
captures sentential entailment and provides ways to analyze the specificity and
preciseness of individual words.

本文定义了能够为句子产生分布的概率模型，最佳表现的模型将每个单词视为线性转换算子应用于多变量高斯分布，并通过从定义的模型中进行训练，演示了实现句子特定性的自然方式。虽然我们的模型是整体最佳表现的，但通过句子向量的范数表示来表示句子的特异性的简单架构也能够表现句子的特异性。质性分析表明，我们的概率模型捕获了句子涵盖并提供了分析单词特异性和精确性的方法。

从释义学习概率句子表示

Learning Probabilistic Sentence Representations from Paraphrases

By representing words with probability densities rather than point vectors,
probabilistic word embeddings can capture rich and interpretable semantic
information and uncertainty. The uncertainty information can be particularly
meaningful in capturing entailment relationships -- whereby general words such
as "entity" correspond to broad distributions that encompass more specific
words such as "animal" or "instrument". We introduce density order embeddings,
which learn hierarchical representations through encapsulation of probability
densities. In particular, we propose simple yet effective loss functions and
distance metrics, as well as graph-based schemes to select negative samples to
better learn hierarchical density representations. Our approach provides
state-of-the-art performance on the WordNet hypernym relationship prediction
task and the challenging HyperLex lexical entailment dataset -- while retaining
a rich and interpretable density representation.

本论文提出了一种基于概率密度的单词嵌入模型 —— 密度顺序嵌入模型。该模型通过实现简单而有效的损失函数和距离度量以及基于图的方案选择负样本，学习到分层的概率密度表示。实验结果表明，此方法在 WordNet 层级关系预测任务和 HyperLex 语义蕴涵数据集上表现出了最先进的性能，并保留了丰富而可解释的密度表示。