Prompt tuning has been an extremely effective tool to adapt a pre-trained model to downstream tasks. However, standard prompt-based methods mainly consider the case of sufficient data of downstream tasks. It is still unclear whether the advantage can be transferred to the few-shot regime, where only limited data are available for each downstream task. Although some works have demonstrated the potential of prompt-tuning under the few-shot setting, the main stream methods via searching discrete prompts or tuning soft prompts with limited data are still very challenging. Through extensive empirical studies, we find that there is still a gap between prompt tuning and fully fine-tuning for few-shot learning. To bridge the gap, we propose a new prompt-tuning framework, called Soft Template Tuning (STT). STT combines manual and auto prompts, and treats downstream classification tasks as a masked language modeling task. Comprehensive evaluation on different settings suggests STT can close the gap between fine-tuning and prompt-based methods without introducing additional parameters. Significantly, it can even outperform the time- and resource-consuming fine-tuning method on sentiment classification tasks.

本研究提出了一种新的 prompt tuning 框架 (Soft Template Tuning，STT)，它综合了手动和自动提示，将下游分类任务视为遮蔽式语言建模任务，并通过大量实证研究表明，STT 可以在不引入额外参数的情况下缩小 fine-tuning 和 prompt-based 方法之间的差距，并且在情感分类任务上甚至可以超越费时和资源消耗更高的 fine-tuning 方法。

STT: 少样本调整的软模板调整