In this paper, we propose a theoretical framework to explain the efficacy of
prompt learning in zero/few-shot scenarios. First, we prove that conventional
pre-training and fine-tuning paradigm fails in few-shot scenarios due to
overfitting the unrepresentative labelled data. We then detail the assumption
that prompt learning is more effective because it empowers pre-trained language
model that is built upon massive text corpora, as well as domain-related human
knowledge to participate more in prediction and thereby reduces the impact of
limited label information provided by the small training set. We further
hypothesize that language discrepancy can measure the quality of prompting.
Comprehensive experiments are performed to verify our assumptions. More
remarkably, inspired by the theoretical framework, we propose an
annotation-agnostic template selection method based on perplexity, which
enables us to ``forecast'' the prompting performance in advance. This approach
is especially encouraging because existing work still relies on development set
to post-hoc evaluate templates. Experiments show that this method leads to
significant prediction benefits compared to state-of-the-art zero-shot methods.

本文提出了一种理论框架，以解释在零 / 少样本场景下提示学习的功效，我们进一步假设语言差异可以衡量提示的质量，并且通过基于 perplexity 的注释无关模板选择方法，使我们能够提前预测提示性能。