Semantic sentence embedding models encode natural language sentences into
vectors, such that closeness in embedding space indicates closeness in the
semantics between the sentences. Bilingual data offers a useful signal for
learning such embeddings: properties shared by both sentences in a translation
pair are likely semantic, while divergent properties are likely stylistic or
language-specific. We propose a deep latent variable model that attempts to
perform source separation on parallel sentences, isolating what they have in
common in a latent semantic vector, and explaining what is left over with
language-specific latent vectors. Our proposed approach differs from past work
on semantic sentence encoding in two ways. First, by using a variational
probabilistic framework, we introduce priors that encourage source separation,
and can use our model's posterior to predict sentence embeddings for
monolingual data at test time. Second, we use high-capacity transformers as
both data generating distributions and inference networks -- contrasting with
most past work on sentence embeddings. In experiments, our approach
substantially outperforms the state-of-the-art on a standard suite of
unsupervised semantic similarity evaluations. Further, we demonstrate that our
approach yields the largest gains on more difficult subsets of these
evaluations where simple word overlap is not a good indicator of similarity.

本文提出了一种基于变分概率框架的深度潜变量模型，使用双语数据，利用拥有共性的语义信息与特性进行训练，使模型可以将并行语句进行源分离，得到隐含的语义向量，进而对单语数据进行预测。实验证明，该方法在无监督语义相似性评估方面的表现明显优于现有方法，并且还能在不适合使用简单词语重叠作为相似性指标的更难评估子集中发挥最大的作用。