Few-shot transfer often shows substantial gain over zero-shot transfer~\cite{lauscher2020zero}, which is a practically useful trade-off between fully supervised and unsupervised learning approaches for multilingual pretrained model-based systems. This paper explores various strategies for selecting data for annotation that can result in a better few-shot transfer. The proposed approaches rely on multiple measures such as data entropy using $n$-gram language model, predictive entropy, and gradient embedding. We propose a loss embedding method for sequence labeling tasks, which induces diversity and uncertainty sampling similar to gradient embedding. The proposed data selection strategies are evaluated and compared for POS tagging, NER, and NLI tasks for up to 20 languages. Our experiments show that the gradient and loss embedding-based strategies consistently outperform random data selection baselines, with gains varying with the initial performance of the zero-shot transfer. Furthermore, the proposed method shows similar trends in improvement even when the model is fine-tuned using a lower proportion of the original task-specific labeled training data for zero-shot transfer.

本文探索了各种数据选择策略，依靠多个度量，利用了渐变嵌入和损失嵌入方法，以提高针对多语言预训练模型的POS标记、NER和NLI任务的Few-shot迁移的性能。实验证明，所提出的方法始终优于随机数据选择基线，即使仅使用更少的标记数据进行零-shot迁移，也能显著提高性能。

多语言少样本转移数据选择的关键是模型中的多样化和不确定性