Non-autoregressive translation (NAT) achieves faster inference speed but at
the cost of worse accuracy compared with autoregressive translation (AT). Since
AT and NAT can share model structure and AT is an easier task than NAT due to
the explicit dependency on previous target-side tokens, a natural idea is to
gradually shift the model training from the easier AT task to the harder NAT
task. To smooth the shift from AT training to NAT training, in this paper, we
introduce semi-autoregressive translation (SAT) as intermediate tasks. SAT
contains a hyperparameter k, and each k value defines a SAT task with different
degrees of parallelism. Specially, SAT covers AT and NAT as its special cases:
it reduces to AT when k = 1 and to NAT when k = N (N is the length of target
sentence). We design curriculum schedules to gradually shift k from 1 to N,
with different pacing functions and number of tasks trained at the same time.
We called our method as task-level curriculum learning for NAT (TCL-NAT).
Experiments on IWSLT14 De-En, IWSLT16 En-De, WMT14 En-De and De-En datasets
show that TCL-NAT achieves significant accuracy improvements over previous NAT
baselines and reduces the performance gap between NAT and AT models to 1-2 BLEU
points, demonstrating the effectiveness of our proposed method.

本研究提出了一种训练曲线，通过任务级别的课程学习（TCL-NAT）将模型训练从更简单的自回归翻译（AT）任务平滑转移到困难的非自回归翻译（NAT）任务，其中引入了中间任务 - 半自回归翻译（SAT），可以通过调整参数 k，平衡翻译速度和准确性，达到在四个数据集上显著提高翻译准确性的效果。