Recent studies have exhibited remarkable capabilities of pre-trained
multilingual Transformers, especially cross-lingual transferability. However,
current methods do not measure cross-lingual transferability well, hindering
the understanding of multilingual Transformers. In this paper, we propose IGap,
a cross-lingual transferability metric for multilingual Transformers on
sentence classification tasks. IGap takes training error into consideration,
and can also estimate transferability without end-task data. Experimental
results show that IGap outperforms baseline metrics for transferability
measuring and transfer direction ranking. Besides, we conduct extensive
systematic experiments where we compare transferability among various
multilingual Transformers, fine-tuning algorithms, and transfer directions.
More importantly, our results reveal three findings about cross-lingual
transfer, which helps us to better understand multilingual Transformers.

本文提出了一种衡量跨语言多语言变压器模型的 IGap 跨语言可转移度度量方法，使用该方法可以更好的理解多语言变压器。实验表明，IGap 在度量可转移性和转移方向排名方面优于基线度量方法，还揭示了跨语言可转移的三个发现。

多语言 Transformer 在句子分类上的跨语言可迁移性测量

Measuring Cross-Lingual Transferability of Multilingual Transformers on  Sentence Classification

In this technique report, we aim to mitigate the overfitting problem of
natural language by applying data augmentation methods. Specifically, we
attempt several types of noise to perturb the input word embedding, such as
Gaussian noise, Bernoulli noise, and adversarial noise, etc. We also apply
several constraints on different types of noise. By implementing these proposed
data augmentation methods, the baseline models can gain improvements on several
sentence classification tasks.

本文提出应用数据增强方法来解决自然语言过拟合问题，尝试通过应用多种噪声扰动输入词嵌入以及在不同类型的噪声上应用几个约束条件，从而改进基线模型在多个句子分类任务上的表现。