In this paper, we introduce XGLUE, a new benchmark dataset that can be used
to train large-scale cross-lingual pre-trained models using multilingual and
bilingual corpora and evaluate their performance across a diverse set of
cross-lingual tasks. Comparing to GLUE(Wang et al., 2019), which is labeled in
English for natural language understanding tasks only, XGLUE has two main
advantages: (1) it provides 11 diversified tasks that cover both natural
language understanding and generation scenarios; (2) for each task, it provides
labeled data in multiple languages. We extend a recent cross-lingual
pre-trained model Unicoder(Huang et al., 2019) to cover both understanding and
generation tasks, which is evaluated on XGLUE as a strong baseline. We also
evaluate the base versions (12-layer) of Multilingual BERT, XLM and XLM-R for
comparison.

本文介绍了 XGLUE 数据集，它是一个可用于训练大规模跨语言预训练模型和评估其在多样的跨语言任务中表现的新的基准数据集。与 GLUE 不同的是，XGLUE 提供了 11 种多样化的任务来覆盖自然语言理解和生成等问题，同时为每个任务提供了多种语言的标注数据。本文还扩展了最近的跨语言预训练模型 Unicoder，以涵盖自然语言理解和生成任务，并在 XGLUE 上作为强基线进行评估。此外，本文还评估了多语言 BERT、XLM 和 XLM-R 的基础版本（12 层）进行比较。