Natural Language Processing systems are heavily dependent on the availability
of annotated data to train practical models. Primarily, models are trained on
English datasets. In recent times, significant advances have been made in
multilingual understanding due to the steeply increasing necessity of working
in different languages. One of the points that stands out is that since there
are now so many pre-trained multilingual models, we can utilize them for
cross-lingual understanding tasks. Using cross-lingual understanding and
Natural Language Inference, it is possible to train models whose applications
extend beyond the training language. We can leverage the power of machine
translation to skip the tiresome part of translating datasets from one language
to another. In this work, we focus on improving the original XNLI dataset by
re-translating the MNLI dataset in all of the 14 different languages present in
XNLI, including the test and dev sets of XNLI using Google Translate. We also
perform experiments by training models in all 15 languages and analyzing their
performance on the task of natural language inference. We then expand our
boundary to investigate if we could improve performance in low-resource
languages such as Swahili and Urdu by training models in languages other than
English.

本篇研究通过重新翻译 14 种不同语言的 MNLI 数据集，包括 XNLI 测试和开发集，以改进原始 XNLI 数据集，并通过在 15 种不同语言中训练模型并分析其在自然语言推断任务上的表现来实现跨语言理解和自然语言处理，同时通过在英语以外的语言中训练模型来探索在资源匮乏的语言（如斯瓦希里和乌尔都语）中提高性能的可能性。

XNLI 2.0：改进 XNLI 数据集和跨语言理解（XLU）性能

XNLI 2.0: Improving XNLI dataset and performance on Cross Lingual Understanding (XLU)

Recent studies have demonstrated the overwhelming advantage of cross-lingual
pre-trained models (PTMs), such as multilingual BERT and XLM, on cross-lingual
NLP tasks. However, existing approaches essentially capture the co-occurrence
among tokens through involving the masked language model (MLM) objective with
token-level cross entropy. In this work, we extend these approaches to learn
sentence-level representations and show the effectiveness on cross-lingual
understanding and generation. Specifically, we propose a Hierarchical
Contrastive Learning (HiCTL) method to (1) learn universal representations for
parallel sentences distributed in one or multiple languages and (2) distinguish
the semantically-related words from a shared cross-lingual vocabulary for each
sentence. We conduct evaluations on two challenging cross-lingual tasks, XTREME
and machine translation. Experimental results show that the HiCTL outperforms
the state-of-the-art XLM-R by an absolute gain of 4.2% accuracy on the XTREME
benchmark as well as achieves substantial improvements on both of the
high-resource and low-resource English-to-X translation tasks over strong
baselines.

本文介绍了一种基于 Hierarchical Contrastive Learning 的方法，用于学习句子级别的跨语言表示方法，在 XTREME 任务和机器翻译任务中实验表明，该方法可以显著提高模型准确性。

跨语言学习通用表征

On Learning Universal Representations Across Languages

The scarcity of labeled training data often prohibits the
internationalization of NLP models to multiple languages. Recent developments
in cross-lingual understanding (XLU) has made progress in this area, trying to
bridge the language barrier using language universal representations. However,
even if the language problem was resolved, models trained in one language would
not transfer to another language perfectly due to the natural domain drift
across languages and cultures. We consider the setting of semi-supervised
cross-lingual understanding, where labeled data is available in a source
language (English), but only unlabeled data is available in the target
language. We combine state-of-the-art cross-lingual methods with recently
proposed methods for weakly supervised learning such as unsupervised
pre-training and unsupervised data augmentation to simultaneously close both
the language gap and the domain gap in XLU. We show that addressing the domain
gap is crucial. We improve over strong baselines and achieve a new
state-of-the-art for cross-lingual document classification.

本文研究跨语言理解中的半监督学习，通过结合最先进的跨语言方法和弱监督学习方法，同时解决语言和领域差异，实现了对跨语言文档分类中的新的领先水平。