Despite the current success of multilingual pre-training, most prior works
focus on leveraging monolingual data or bilingual parallel data and overlooked
the value of trilingual parallel data. This paper presents \textbf{Tri}angular
Document-level \textbf{P}re-training (\textbf{TRIP}), which is the first in the
field to extend the conventional monolingual and bilingual pre-training to a
trilingual setting by (i) \textbf{Grafting} the same documents in two languages
into one mixed document, and (ii) predicting the remaining one language as the
reference translation. Our experiments on document-level MT and cross-lingual
abstractive summarization show that TRIP brings by up to 3.65 d-BLEU points and
6.2 ROUGE-L points on three multilingual document-level machine translation
benchmarks and one cross-lingual abstractive summarization benchmark, including
multiple strong state-of-the-art (SOTA) scores. In-depth analysis indicates
that TRIP improves document-level machine translation and captures better
document contexts in at least three characteristics: (i) tense consistency,
(ii) noun consistency and (iii) conjunction presence.

本文介绍了第一个将常规的单语和双语预训练扩展到三语环境的三角形文档级预训练（TRIP）方法，在对文档级机器翻译和跨语言抽象摘要的实验中，TRIP 在三项多语种文档级机器翻译基准和一个跨语言抽象摘要基准上提高了多达 3.65 d-BLEU 点和 6.2 ROUGE-L 点，并且包括多个强大的最先进得分状态。

TRIP: 多语言语言模型三角形式的文档级预训练

TRIP: Triangular Document-level Pre-training for Multilingual Language Models

Cross-lingual summarization (CLS) is the task to produce a summary in one
particular language for a source document in a different language. We introduce
WikiMulti - a new dataset for cross-lingual summarization based on Wikipedia
articles in 15 languages. As a set of baselines for further studies, we
evaluate the performance of existing cross-lingual abstractive summarization
methods on our dataset. We make our dataset publicly available here:
this https URL

介绍了基于维基百科 15 种语言文章的新日数据集 - WikiMulti， 用于跨语言摘要生成 (CLS) 任务，并在该数据集上评价了现有的跨语言抽象性摘要方法的性能。

WikiMulti: 用于跨语言摘要的语料库

WikiMulti: a Corpus for Cross-Lingual Summarization

We present CrossSum, a large-scale cross-lingual abstractive summarization
dataset comprising 1.7 million article-summary samples in 1500+ language pairs.
We create CrossSum by aligning identical articles written in different
languages via cross-lingual retrieval from a multilingual summarization
dataset. We propose a multi-stage data sampling algorithm to effectively train
a cross-lingual summarization model capable of summarizing an article in any
target language. We also propose LaSE, a new metric for automatically
evaluating model-generated summaries and showing a strong correlation with
ROUGE. Performance on ROUGE and LaSE indicate that pretrained models fine-tuned
on CrossSum consistently outperform baseline models, even when the source and
target language pairs are linguistically distant. To the best of our knowledge,
CrossSum is the largest cross-lingual summarization dataset and the first-ever
that does not rely solely on English as the pivot language. We are releasing
the dataset, alignment and training scripts, and the models to spur future
research on cross-lingual abstractive summarization. The resources can be found
at this https URL

通过跨语言检索，提出跨语言摘要数据集 CrossSum 及其训练模型，表现出优异的 ROUGE 和 LaSE 性能，成为目前最大的跨语言摘要数据集

CrossSum: 跨越 1500 多种语言对的英语中心的跨语言生成性文本摘要

CrossSum: Beyond English-Centric Cross-Lingual Abstractive Text  Summarization for 1500+ Language Pairs

We present a multi-task learning framework for cross-lingual abstractive
summarization to augment training data. Recent studies constructed pseudo
cross-lingual abstractive summarization data to train their neural
encoder-decoders. Meanwhile, we introduce existing genuine data such as
translation pairs and monolingual abstractive summarization data into training.
Our proposed method, Transum, attaches a special token to the beginning of the
input sentence to indicate the target task. The special token enables us to
incorporate the genuine data into the training data easily. The experimental
results show that Transum achieves better performance than the model trained
with only pseudo cross-lingual summarization data. In addition, we achieve the
top ROUGE score on Chinese-English and Arabic-English abstractive
summarization. Moreover, Transum also has a positive effect on machine
translation. Experimental results indicate that Transum improves the
performance from the strong baseline, Transformer, in Chinese-English,
Arabic-English, and English-Japanese translation datasets.

Transum 是一种多任务学习框架，可以将真实数据及伪造数据结合起来用于神经编码器 - 解码器的训练，达到在汉英和阿拉伯英文摘要、中英、阿拉伯英翻译任务中均获得最佳 ROUGE 分的效果，此外还对机器翻译任务有积极的影响。