This work investigates the alignment problem in state-of-the-art multi-head
attention models based on the transformer architecture. We demonstrate that
alignment extraction in transformer models can be improved by augmenting an
additional alignment head to the multi-head source-to-target attention
component. This is used to compute sharper attention weights. We describe how
to use the alignment head to achieve competitive performance. To study the
effect of adding the alignment head, we simulate a dictionary-guided
translation task, where the user wants to guide translation using pre-defined
dictionary entries. Using the proposed approach, we achieve up to $3.8$ % BLEU
improvement when using the dictionary, in comparison to $2.4$ % BLEU in the
baseline case. We also propose alignment pruning to speed up decoding in
alignment-based neural machine translation (ANMT), which speeds up translation
by a factor of $1.8$ without loss in translation performance. We carry out
experiments on the shared WMT 2016 English$\to$Romanian news task and the BOLT
Chinese$\to$English discussion forum task.

本研究旨在调查基于 Transformer 架构的多头注意力模型中的对齐问题，并证明了通过增加一个额外的对齐头可以提高 Transformer 模型的对齐抽取。通过字典引导翻译任务的模拟，我们使用所提出的方法，在与基准情况相比，使用字典时 BLEU 分数提高了 3.8％，而基准情况下提高了 2.4％的 BLEU 分数。我们还提出了对齐剪枝来加速解码，从而在保持翻译性能不变的情况下，将翻译加速了 1.8 倍。