Supervised fine-tuning (SFT), supervised instruction tuning (SIT) and
in-context learning (ICL) are three alternative, de facto standard approaches
to few-shot learning. ICL has gained popularity recently with the advent of
LLMs due to its simplicity and sample efficiency. Prior research has conducted
only limited investigation into how these approaches work for multilingual
few-shot learning, and the focus so far has been mostly on their performance.
In this work, we present an extensive and systematic comparison of the three
approaches, testing them on 6 high- and low-resource languages, three different
NLU tasks, and a myriad of language and domain setups. Importantly, performance
is only one aspect of the comparison, where we also analyse the approaches
through the optics of their computational, inference and financial costs. Our
observations show that supervised instruction tuning has the best trade-off
between performance and resource requirements. As another contribution, we
analyse the impact of target language adaptation of pretrained LLMs and find
that the standard adaptation approaches can (superficially) improve target
language generation capabilities, but language understanding elicited through
ICL does not improve and remains limited, with low scores especially for
low-resource languages.

三种常用方法，即监督微调、监督指令微调和上下文学习，是少样本学习的三种替代方法。本文对这三种方法进行了广泛而系统的比较，测试了六种高低资源语言、三种不同的 NLU 任务以及各种语言和领域设置。观察结果显示，监督指令微调在性能和资源需求方面具有最佳平衡性。此外，本文还分析了预训练 LLM 的目标语言适应性，并发现标准适应方法能在表面上提高目标语言生成能力，但经 ICL 引发的语言理解并未改善且受限，尤其是对于低资源语言而言，得分较低。