Despite the current success of multilingual pre-training, most prior works
focus on leveraging monolingual data or bilingual parallel data and overlooked
the value of trilingual parallel data. This paper presents \textbf{Tri}angular
Document-level \textbf{P}re-training (\textbf{TRIP}), which is the first in the
field to extend the conventional monolingual and bilingual pre-training to a
trilingual setting by (i) \textbf{Grafting} the same documents in two languages
into one mixed document, and (ii) predicting the remaining one language as the
reference translation. Our experiments on document-level MT and cross-lingual
abstractive summarization show that TRIP brings by up to 3.65 d-BLEU points and
6.2 ROUGE-L points on three multilingual document-level machine translation
benchmarks and one cross-lingual abstractive summarization benchmark, including
multiple strong state-of-the-art (SOTA) scores. In-depth analysis indicates
that TRIP improves document-level machine translation and captures better
document contexts in at least three characteristics: (i) tense consistency,
(ii) noun consistency and (iii) conjunction presence.

本文介绍了第一个将常规的单语和双语预训练扩展到三语环境的三角形文档级预训练（TRIP）方法，在对文档级机器翻译和跨语言抽象摘要的实验中，TRIP 在三项多语种文档级机器翻译基准和一个跨语言抽象摘要基准上提高了多达 3.65 d-BLEU 点和 6.2 ROUGE-L 点，并且包括多个强大的最先进得分状态。