Non-Autoregressive Neural Machine Translation (NAT) achieves significant
decoding speedup through generating target words independently and
simultaneously. However, in the context of non-autoregressive translation, the
word-level cross-entropy loss cannot model the target-side sequential
dependency properly, leading to its weak correlation with the translation
quality. As a result, NAT tends to generate influent translations with
over-translation and under-translation errors. In this paper, we propose to
train NAT to minimize the Bag-of-Ngrams (BoN) difference between the model
output and the reference sentence. The bag-of-ngrams training objective is
differentiable and can be efficiently calculated, which encourages NAT to
capture the target-side sequential dependency and correlates well with the
translation quality. We validate our approach on three translation tasks and
show that our approach largely outperforms the NAT baseline by about 5.0 BLEU
scores on WMT14 En$\leftrightarrow$De and about 2.5 BLEU scores on WMT16
En$\leftrightarrow$Ro.

本文提出通过训练 NAT 最小化模型输出和参考句子之间的 N-Gram 差异来促进 NAT 捕捉目标方面的序列依赖性，并与翻译质量相关。在三个翻译任务上验证了我们的方法，结果显示我们的方法在 WMT14 En<->De 和 WMT16 En<->Ro 上比 NAT 基线大约 5.0 和 2.5 个 BLEU 分数。