Non-autoregressive neural machine translation (NAT) models suffer from the
multi-modality problem that there may exist multiple possible translations of a
source sentence, so the reference sentence may be inappropriate for the
training when the NAT output is closer to other translations. In response to
this problem, we introduce a rephraser to provide a better training target for
NAT by rephrasing the reference sentence according to the NAT output. As we
train NAT based on the rephraser output rather than the reference sentence, the
rephraser output should fit well with the NAT output and not deviate too far
from the reference, which can be quantified as reward functions and optimized
by reinforcement learning. Experiments on major WMT benchmarks and NAT
baselines show that our approach consistently improves the translation quality
of NAT. Specifically, our best variant achieves comparable performance to the
autoregressive Transformer, while being 14.7 times more efficient in inference.

本文介绍了一种针对非自回归神经机器翻译（NAT）多模问题的解决方案：引入一种重新表述器 (rephraser) 来为 NAT 提供更好的训练目标，并通过强化学习优化该目标，从而提高 NAT 翻译质量，在主要 WMT 基准和 NAT 基线上实验表明相比原始 NAT 的翻译质量有了明显的提高，且最优实验方案的推理效率是自回归 Transformer 的 14.7 倍。