Recently, non-autoregressive (NAR) neural machine translation models have
received increasing attention due to their efficient parallel decoding.
However, the probabilistic framework of NAR models necessitates conditional
independence assumption on target sequences, falling short of characterizing
human language data. This drawback results in less informative learning signals
for NAR models under conventional MLE training, thereby yielding unsatisfactory
accuracy compared to their autoregressive (AR) counterparts. In this paper, we
propose a simple and model-agnostic multi-task learning framework to provide
more informative learning signals. During training stage, we introduce a set of
sufficiently weak AR decoders that solely rely on the information provided by
NAR decoder to make prediction, forcing the NAR decoder to become stronger or
else it will be unable to support its weak AR partners. Experiments on WMT and
IWSLT datasets show that our approach can consistently improve accuracy of
multiple NAR baselines without adding any additional decoding overhead.

提出一种多任务学习框架，加强非自回归神经机器翻译模型的学习信号并改进其准确性。实验证明该方法可以在不增加解码开销的情况下稳定提升多个非自回归基准模型的准确性。