How can we extend a pre-trained model to many language understanding tasks,
without labeled or additional unlabeled data? Pre-trained language models
(PLMs) have been effective for a wide range of NLP tasks. However, existing
approaches either require fine-tuning on downstream labeled datasets or
manually constructing proper prompts. In this paper, we propose nonparametric
prompting PLM (NPPrompt) for fully zero-shot language understanding. Unlike
previous methods, NPPrompt uses only pre-trained language models and does not
require any labeled data or additional raw corpus for further fine-tuning, nor
does it rely on humans to construct a comprehensive set of prompt label words.
We evaluate NPPrompt against previous major few-shot and zero-shot learning
methods on diverse NLP tasks: including text classification, text entailment,
similar text retrieval, and paraphrasing. Experimental results demonstrate that
our NPPrompt outperforms the previous best fully zero-shot method by big
margins, with absolute gains of 12.8% in accuracy on text classification and
18.9% on the GLUE benchmark.

本文提出了一种基于预训练语言模型的无参考学习方法 NPPrompt，能够有效扩展一个预训练模型到多个语言理解任务，且不需要标注数据或附加未标注语料库进行微调。实验结果表明，NPPrompt 在文本分类和 GLUE 基准测试等任务中具有比以前最好的全零样本方法更高的绝对增益。