Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning. In this work, we show that finetuning LMs in the few-shot setting can considerably reduce the need for prompt engineering. In fact, one can use null prompts, prompts that contain neither task-specific templates nor training examples, and achieve competitive accuracy to manually-tuned prompts across a wide range of tasks. While finetuning LMs does introduce new parameters for each downstream task, we show that this memory overhead can be substantially reduced: finetuning only the bias terms can achieve comparable or better accuracy than standard finetuning while only updating 0.1% of the parameters. All in all, we recommend finetuning LMs for few-shot learning as it is more accurate, robust to different prompts, and can be made nearly as efficient as using frozen LMs.

使用少量训练示例和任务说明来训练语言模型对于几乎所有任务都很重要，本文提出在极小数据量情境下调整 LM 可显著降低提示工程需求，使用 0.1% 参数更新的 bias terms 可以实现与标准调整相当甚至更好的准确性。

减少提示和参数：使用语言模型进行简单的小样本学习