Recent years have witnessed the rapid advance in neural machine translation
(NMT), the core of which lies in the encoder-decoder architecture. Inspired by
the recent progress of large-scale pre-trained language models on machine
translation in a limited scenario, we firstly demonstrate that a single
language model (LM4MT) can achieve comparable performance with strong
encoder-decoder NMT models on standard machine translation benchmarks, using
the same training data and similar amount of model parameters. LM4MT can also
easily utilize source-side texts as additional supervision. Though modeling the
source- and target-language texts with the same mechanism, LM4MT can provide
unified representations for both source and target sentences, which can better
transfer knowledge across languages. Extensive experiments on pivot-based and
zero-shot translation tasks show that LM4MT can outperform the encoder-decoder
NMT model by a large margin.

该研究表明，预训练的单一语言模型（LM4MT）可以与强大的编码器 - 解码器 NMT 模型在标准机器翻译基准测试中取得可比较的性能，LM4MT 还可以轻松利用源侧文本作为额外的监督。LM4MT 可以为源语言和目标语言提供统一的表示形式，在跨语言知识转移方面表现更好，并在基于中间语言和零样本翻译任务中获得显著的优势。