Recent foundational language models have shown state-of-the-art performance
in many NLP tasks in zero- and few-shot settings. An advantage of these models
over more standard approaches based on fine-tuning is the ability to understand
instructions written in natural language (prompts), which helps them generalise
better to different tasks and domains without the need for specific training
data. This makes them suitable for addressing text classification problems for
domains with limited amounts of annotated instances. However, existing research
is limited in scale and lacks understanding of how text generation models
combined with prompting techniques compare to more established methods for text
classification such as fine-tuning masked language models. In this paper, we
address this research gap by performing a large-scale evaluation study for 16
text classification datasets covering binary, multiclass, and multilabel
problems. In particular, we compare zero- and few-shot approaches of large
language models to fine-tuning smaller language models. We also analyse the
results by prompt, classification type, domain, and number of labels. In
general, the results show how fine-tuning smaller and more efficient language
models can still outperform few-shot approaches of larger language models,
which have room for improvement when it comes to text classification.

在这篇论文中，我们通过对 16 个文本分类数据集进行大规模评估研究，比较了零样本和少样本的大型语言模型与微调较小语言模型在文本分类方面的表现。结果表明，更小且更高效的语言模型的微调仍然能胜过大型语言模型的少样本方法，在文本分类方面有改进的空间。