Word alignment is to find translationally equivalent words between source and
target sentences. Previous work has demonstrated that self-training can achieve
competitive word alignment results. In this paper, we propose to use word
alignments generated by a third-party word aligner to supervise the neural word
alignment training. Specifically, source word and target word of each word pair
aligned by the third-party aligner are trained to be close neighbors to each
other in the contextualized embedding space when fine-tuning a pre-trained
cross-lingual language model. Experiments on the benchmarks of various language
pairs show that our approach can surprisingly do self-correction over the
third-party supervision by finding more accurate word alignments and deleting
wrong word alignments, leading to better performance than various third-party
word aligners, including the currently best one. When we integrate all
supervisions from various third-party aligners, we achieve state-of-the-art
word alignment performances, with averagely more than two points lower
alignment error rates than the best third-party aligner. We released our code
at this https URL.

本文提出使用第三方词对齐器生成的单词对齐来监督神经单词对齐训练，在微调预训练的跨语言语言模型时，目标是让每个单词对的源语言单词和目标语言单词在情境化的嵌入空间中成为彼此相邻的接近邻居，实现自我训练并在各种语言对的基准测试中获得最先进的单词对齐性能。