Multilingual Pretrained Language Models (MPLMs) have shown their strong
multilinguality in recent empirical cross-lingual transfer studies. In this
paper, we propose the Prompts Augmented by Retrieval Crosslingually (PARC)
pipeline to improve the zero-shot performance on low-resource languages (LRLs)
by augmenting the context with semantically similar sentences retrieved from a
high-resource language (HRL) as prompts. PARC improves the zero-shot
performance on three downstream tasks (binary sentiment classification, topic
categorization and natural language inference) with multilingual parallel test
sets across 10 LRLs covering 6 language families in both unlabeled settings
(+5.1%) and labeled settings (+16.3%). PARC-labeled also outperforms the
finetuning baseline by 3.7%. We find a significant positive correlation between
cross-lingual transfer performance on one side, and the similarity between the
high- and low-resource languages as well as the amount of low-resource
pretraining data on the other side. A robustness analysis suggests that PARC
has the potential to achieve even stronger performance with more powerful
MPLMs.

使用高资源语言作为提示，通过检索从而增强上下文，提出的 PARC 管道，可以提高低资源语言的零 - shot 表现。在对涵盖 6 个语言家族的 10 个低资源语言进行多语言并行测试集时，PARC 可以在未标记的设置中提高 5.1％，在标记的设置中提高 16.3％。同时还比微调基线的效果提高了 3.7％。