Attention-based Neural Machine Translation (NMT) models suffer from attention
deficiency issues as has been observed in recent research. We propose a novel
mechanism to address some of these limitations and improve the NMT attention.
Specifically, our approach memorizes the alignments temporally (within each
sentence) and modulates the attention with the accumulated temporal memory, as
the decoder generates the candidate translation. We compare our approach
against the baseline NMT model and two other related approaches that address
this issue either explicitly or implicitly. Large-scale experiments on two
language pairs show that our approach achieves better and robust gains over the
baseline and related NMT approaches. Our model further outperforms strong SMT
baselines in some settings even without using ensembles.

本文介绍了一种基于注意力机制的神经机器翻译模型，该模型通过暂时性存储对齐信息和调节注意力值的方法，提高了翻译效果， 在两种语言对上的大规模实验中，该方法不仅超越了基线 NMT 模型，还比其他相关方法表现更出色，甚至有些情况下在不使用集成的情况下也能胜过 SMT 基准。