Word alignment, which aims to align translationally equivalent words between
source and target sentences, plays an important role in many natural language
processing tasks. Current unsupervised neural alignment methods focus on
inducing alignments from neural machine translation models, which does not
leverage the full context in the target sequence. In this paper, we propose
Mask-Align, a self-supervised word alignment model that takes advantage of the
full context on the target side. Our model masks out each target token and
predicts it conditioned on both source and the remaining target tokens. This
two-step process is based on the assumption that the source token contributing
most to recovering the masked target token should be aligned. We also introduce
an attention variant called leaky attention, which alleviates the problem of
unexpected high cross-attention weights on special tokens such as periods.
Experiments on four language pairs show that our model outperforms previous
unsupervised neural aligners and obtains new state-of-the-art results.

本研究提出一种自监督的单词对齐模型 - Mask-Align，该模型利用目标序列上的完整上下文，在进行两步的处理，同时采用一种叫做漏洞关注 (leaky attention) 的注意力变体，对于特殊符号如句号，该变体有助于缓解跨项权重过高的问题，各项结果显示，该模型的性能优于以前的无人监督神经对齐器，同时取得了新的最优结果。