In recent years, Neural Machine Translation (NMT) has achieved notable
results in various translation tasks. However, the word-by-word generation
manner determined by the autoregressive mechanism leads to high translation
latency of the NMT and restricts its low-latency applications.
Non-Autoregressive Neural Machine Translation (NAT) removes the autoregressive
mechanism and achieves significant decoding speedup through generating target
words independently and simultaneously. Nevertheless, NAT still takes the
word-level cross-entropy loss as the training objective, which is not optimal
because the output of NAT cannot be properly evaluated due to the multimodality
problem. In this article, we propose using sequence-level training objectives
to train NAT models, which evaluate the NAT outputs as a whole and correlates
well with the real translation quality. Firstly, we propose training NAT models
to optimize sequence-level evaluation metrics (e.g., BLEU) based on several
novel reinforcement algorithms customized for NAT, which outperforms the
conventional method by reducing the variance of gradient estimation. Secondly,
we introduce a novel training objective for NAT models, which aims to minimize
the Bag-of-Ngrams (BoN) difference between the model output and the reference
sentence. The BoN training objective is differentiable and can be calculated
efficiently without doing any approximations. Finally, we apply a three-stage
training strategy to combine these two methods to train the NAT model. We
validate our approach on four translation tasks (WMT14 En$\leftrightarrow$De,
WMT16 En$\leftrightarrow$Ro), which shows that our approach largely outperforms
NAT baselines and achieves remarkable performance on all translation tasks. The
source code is available at this https URL

本文提出了一种使用序列级训练目标来训练非自回归神经机器翻译模型的方法，通过几种为非自回归定制的新型强化算法，基于 BLEU 等序列级评价指标优化 NAT 模型的训练；介绍了一种基于 Bag-of-Ngrams（BoN）差异的 NAT 模型的新型训练目标；并通过三阶段训练策略将这两种方法结合起来应用，验证了该方法在多项任务中的鲜明表现。