Multilingual pretrained language models (mPLMs) have shown their
effectiveness in multilingual word alignment induction. However, these methods
usually start from mBERT or XLM-R. In this paper, we investigate whether
multilingual sentence Transformer LaBSE is a strong multilingual word aligner.
This idea is non-trivial as LaBSE is trained to learn language-agnostic
sentence-level embeddings, while the alignment extraction task requires the
more fine-grained word-level embeddings to be language-agnostic. We demonstrate
that the vanilla LaBSE outperforms other mPLMs currently used in the alignment
task, and then propose to finetune LaBSE on parallel corpus for further
improvement. Experiment results on seven language pairs show that our best
aligner outperforms previous state-of-the-art models of all varieties. In
addition, our aligner supports different language pairs in a single model, and
even achieves new state-of-the-art on zero-shot language pairs that does not
appear in the finetuning process.

本篇论文探究了多语言句子 Transformer LaBSE 在多语言单词对齐问题上的表现以及提出了一种用于进一步改进的平行语料库微调方法，实验证明我们的方法优于先前的最先进模型，可支持单个模型中的不同语言对的对齐，甚至在零射击语言对上实现了新的最先进。