Context gates are effective to control the contributions from the source and
target contexts in the recurrent neural network (RNN) based neural machine
translation (NMT). However, it is challenging to extend them into the advanced
Transformer architecture, which is more complicated than RNN. This paper first
provides a method to identify source and target contexts and then introduce a
gate mechanism to control the source and target contributions in Transformer.
In addition, to further reduce the bias problem in the gate mechanism, this
paper proposes a regularization method to guide the learning of the gates with
supervision automatically generated using pointwise mutual information.
Extensive experiments on 4 translation datasets demonstrate that the proposed
model obtains an averaged gain of 1.0 BLEU score over a strong Transformer
baseline.

本文提出了一种在 Transformer 架构中引入 context gates 的方法，以控制源和目标的贡献。此外，还提出了一种规范化方法，以自动产生的点间互信息来指导门控的学习。在 4 个翻译数据集上的实验证明，该模型相对于强基线模型提高 1.0 的 BLEU 分数。

正则化上下文门控制在 Transformer 机器翻译中的应用

Regularized Context Gates on Transformer for Machine Translation

In neural machine translation (NMT), generation of a target word depends on
both source and target contexts. We find that source contexts have a direct
impact on the adequacy of a translation while target contexts affect the
fluency. Intuitively, generation of a content word should rely more on the
source context and generation of a functional word should rely more on the
target context. Due to the lack of effective control over the influence from
source and target contexts, conventional NMT tends to yield fluent but
inadequate translations. To address this problem, we propose context gates
which dynamically control the ratios at which source and target contexts
contribute to the generation of target words. In this way, we can enhance both
the adequacy and fluency of NMT with more careful control of the information
flow from contexts. Experiments show that our approach significantly improves
upon a standard attention-based NMT system by +2.3 BLEU points.

为解决传统神经机器翻译技术所存在的译文质量缺陷问题，我们提出了一种名为 context gates 的动态控制算法，旨在更加细致地控制源与目标语境对目标词汇生成的影响比率，实验结果显示较传统注意力机制神经机器翻译系统提升 2.3 BLEU 分数。