This work presents methods for learning cross-lingual sentence
representations using paired or unpaired bilingual texts. We hypothesize that
the cross-lingual alignment strategy is transferable, and therefore a model
trained to align only two languages can encode multilingually more aligned
representations. We thus introduce dual-pivot transfer: training on one
language pair and evaluating on other pairs. To study this theory, we design
unsupervised models trained on unpaired sentences and single-pair supervised
models trained on bitexts, both based on the unsupervised language model XLM-R
with its parameters frozen. The experiments evaluate the models as universal
sentence encoders on the task of unsupervised bitext mining on two datasets,
where the unsupervised model reaches the state of the art of unsupervised
retrieval, and the alternative single-pair supervised model approaches the
performance of multilingually supervised models. The results suggest that
bilingual training techniques as proposed can be applied to get sentence
representations with multilingual alignment.

本文提出一种跨语言句子表示的学习方法，介绍了双轴转移的训练策略，并通过实验表明这种策略可以应用于获取具有多语言对齐的句子表示。