Previous work mainly focuses on improving cross-lingual transfer for NLU
tasks with a multilingual pretrained encoder (MPE), or improving the
performance on supervised machine translation with BERT. However, it is
under-explored that whether the MPE can help to facilitate the cross-lingual
transferability of NMT model. In this paper, we focus on a zero-shot
cross-lingual transfer task in NMT. In this task, the NMT model is trained with
parallel dataset of only one language pair and an off-the-shelf MPE, then it is
directly tested on zero-shot language pairs. We propose SixT, a simple yet
effective model for this task. SixT leverages the MPE with a two-stage training
schedule and gets further improvement with a position disentangled encoder and
a capacity-enhanced decoder. Using this method, SixT significantly outperforms
mBART, a pretrained multilingual encoder-decoder model explicitly designed for
NMT, with an average improvement of 7.1 BLEU on zero-shot any-to-English test
sets across 14 source languages. Furthermore, with much less training
computation cost and training data, our model achieves better performance on 15
any-to-English test sets than CRISS and m2m-100, two strong multilingual NMT
baselines.

本研究提出了 SixT，一种利用多语言预训练编码器（MPE）来促进神经机器翻译模型的跨语言迁移的方法。通过采用两阶段训练计划和位置解缠编码器以及容量增强解码器，SixT 在零基础多语言翻译任务上取得了比已有模型更好的结果。