Contrastive learning has been successfully used for retrieval of semantically
aligned sentences, but it often requires large batch sizes or careful
engineering to work well. In this paper, we instead propose a generative model
for learning multilingual text embeddings which can be used to retrieve or
score sentence pairs. Our model operates on parallel data in $N$ languages and,
through an approximation we introduce, efficiently encourages source separation
in this multilingual setting, separating semantic information that is shared
between translations from stylistic or language-specific variation. We show
careful large-scale comparisons between contrastive and generation-based
approaches for learning multilingual text embeddings, a comparison that has not
been done to the best of our knowledge despite the popularity of these
approaches. We evaluate this method on a suite of tasks including semantic
similarity, bitext mining, and cross-lingual question retrieval -- the last of
which we introduce in this paper. Overall, our Variational Multilingual
Source-Separation Transformer (VMSST) model outperforms both a strong
contrastive and generative baseline on these tasks.

本文提出了一种新的多语种文本嵌入生成模型 —— VMSST，通过在 $N$ 种语言的并行数据上运行，通过一个引入的近似算法，在多语种语境中鼓励源分离，我们在比较学习多语种文本嵌入的对比和基于生成的方法时，对其进行了仔细的大规模比较，并对其进行了语义相似性、位文本挖掘、跨语言问题检索等任务的评估，从而证明了其优越性。