Zero-shot cross-lingual transfer is a central task in multilingual NLP,
allowing models trained in languages with more sufficient training resources to
generalize to other low-resource languages. Earlier efforts on this task use
parallel corpora, bilingual dictionaries, or other annotated alignment data to
improve cross-lingual transferability, which are typically expensive to obtain.
In this paper, we propose a simple yet effective method, SALT, to improve the
zero-shot cross-lingual transfer of the multilingual pretrained language models
without the help of such external data. By incorporating code-switching and
embedding mixup with self-augmentation, SALT effectively distills cross-lingual
knowledge from the multilingual PLM and enhances its transferability on
downstream tasks. Experimental results on XNLI and PAWS-X show that our method
is able to improve zero-shot cross-lingual transferability without external
data. Our code is available at this https URL

提出一种称为 SALT 的简单而有效的方法，结合了代码混合和嵌入混合自增强，通过从多语言预训练语言模型中提取跨语言知识并增强其在下游任务中的可转移性，改进了零射击跨语言转移能力，而无需外部数据。