Fully non-autoregressive neural machine translation (NAT) is proposed to
simultaneously predict tokens with single forward of neural networks, which
significantly reduces the inference latency at the expense of quality drop
compared to the Transformer baseline. In this work, we target on closing the
performance gap while maintaining the latency advantage. We first inspect the
fundamental issues of fully NAT models, and adopt dependency reduction in the
learning space of output tokens as the basic guidance. Then, we revisit methods
in four different aspects that have been proven effective for improving NAT
models, and carefully combine these techniques with necessary modifications.
Our extensive experiments on three translation benchmarks show that the
proposed system achieves the new state-of-the-art results for fully NAT models,
and obtains comparable performance with the autoregressive and iterative NAT
systems. For instance, one of the proposed models achieves 27.49 BLEU points on
WMT14 En-De with approximately 16.5X speed up at inference time.

本研究提出了一种全非自回归神经机器翻译（NAT）的方案，采用依赖关系减少等方法，缩短推理延迟同时提升翻译质量，在三个翻译基准测试中实现了新的全自然机器翻译模型的最高水平，并在推理时间上取得了大约 16.5 倍的速度提升。