Training or finetuning large-scale language models (LLMs) such as GPT-3
requires substantial computation resources, motivating recent efforts to
explore parameter-efficient adaptation to downstream tasks. One practical area
of research is to treat these models as black boxes and interact with them
through their inference APIs. In this paper, we investigate how to optimize
few-shot text classification without accessing the gradients of the LLMs. To
achieve this, we treat the black-box model as a feature extractor and train a
classifier with the augmented text data. Data augmentation is performed using
prompt-based finetuning on an auxiliary language model with a much smaller
parameter size than the black-box model. Through extensive experiments on eight
text classification datasets, we show that our approach, dubbed BT-Classifier,
significantly outperforms state-of-the-art black-box few-shot learners and
performs on par with methods that rely on full-model tuning.

本文提出了一种名为 BT-Classifier 的方法，它通过对数据进行数据增强、把黑盒模型作为特征提取器以及使用小型的辅助语言模型的 prompt-based 微调训练分类器，达到了在不访问大型模型的梯度的情况下优化零样本学习任务的效果，并在 8 个文本分类数据集上取得了与全模型调整方法相当的成果。