Standard Full-Data classifiers in NLP demand thousands of labeled examples,
which is impractical in data-limited domains. Few-shot methods offer an
alternative, utilizing contrastive learning techniques that can be effective
with as little as 20 examples per class. Similarly, Large Language Models
(LLMs) like GPT-4 can perform effectively with just 1-5 examples per class.
However, the performance-cost trade-offs of these methods remain underexplored,
a critical concern for budget-limited organizations. Our work addresses this
gap by studying the aforementioned approaches over the Banking77 financial
intent detection dataset, including the evaluation of cutting-edge LLMs by
OpenAI, Cohere, and Anthropic in a comprehensive set of few-shot scenarios. We
complete the picture with two additional methods: first, a cost-effective
querying method for LLMs based on retrieval-augmented generation (RAG), able to
reduce operational costs multiple times compared to classic few-shot
approaches, and second, a data augmentation method using GPT-4, able to improve
performance in data-limited scenarios. Finally, to inspire future research, we
provide a human expert's curated subset of Banking77, along with extensive
error analysis.

标准的 NLP 全数据分类器需要成千上万个标记示例，在数据有限的领域中是不切实际的。少样本学习方法提供了一种替代方案，利用对比学习技术，在每个类别只需 20 个示例即可发挥有效作用。类似地，像 GPT-4 这样的大型语言模型也可在每个类别只有 1-5 个示例的情况下有效运行。然而，这些方法的性能与成本之间的权衡仍然未被充分研究，这对于预算有限的组织来说是一个关键问题。我们的工作通过在 Banking77 金融意图检测数据集上研究上述方法来填补这一空白，包括评估 OpenAI、Cohere 和 Anthropic 的最新大型语言模型在全面的少样本场景中的表现。我们通过两种额外的方法来完善整个研究：一是基于检索增强生成（RAG）的成本效益查询方法，与经典的少样本方法相比，能够多次降低运营成本；二是使用 GPT-4 的数据增强方法，能够改善数据有限情况下的性能。最后，为了激发未来的研究，我们提供了一个人工专家策划的 Banking77 子集，以及广泛的错误分析。