Existing models of multilingual sentence embeddings require large parallel
data resources which are not available for low-resource languages. We propose a
novel unsupervised method to derive multilingual sentence embeddings relying
only on monolingual data. We first produce a synthetic parallel corpus using
unsupervised machine translation, and use it to fine-tune a pretrained
cross-lingual masked language model (XLM) to derive the multilingual sentence
representations. The quality of the representations is evaluated on two
parallel corpus mining tasks with improvements of up to 22 F1 points over
vanilla XLM. In addition, we observe that a single synthetic bilingual corpus
is able to improve results for other language pairs.

本研究提出了一种新的无监督方法，通过使用单语数据来获得跨语言句子嵌入，产生了合成平行语料库，使用预训练的跨语言掩码语言模型（XLM）对其进行微调以得到多语言句子表示，并在两个平行语料库挖掘任务上评估了表示的质量，结果表明，这种方法可以比基准 XLM 模型获得高达 22 个 F1 点的改进。此外，我们还观察到，单个合成的双语语料库能够改善其他语言对的结果。