Recent advances on prompt-tuning cast few-shot classification tasks as a
masked language modeling problem. By wrapping input into a template and using a
verbalizer which constructs a mapping between label space and label word space,
prompt-tuning can achieve excellent results in zero-shot and few-shot
scenarios. However, typical prompt-tuning needs a manually designed verbalizer
which requires domain expertise and human efforts. And the insufficient label
space may introduce considerable bias into the results. In this paper, we focus
on eliciting knowledge from pretrained language models and propose a
prototypical prompt verbalizer for prompt-tuning. Labels are represented by
prototypical embeddings in the feature space rather than by discrete words. The
distances between the embedding at the masked position of input and
prototypical embeddings are used as classification criterion. For zero-shot
settings, knowledge is elicited from pretrained language models by a manually
designed template to form initial prototypical embeddings. For few-shot
settings, models are tuned to learn meaningful and interpretable prototypical
embeddings. Our method optimizes models by contrastive learning. Extensive
experimental results on several many-class text classification datasets with
low-resource settings demonstrate the effectiveness of our approach compared
with other verbalizer construction methods. Our implementation is available at
this https URL.

本研究探讨了如何从预训练语言模型中提取知识，提出了一种基于原型的提示语言生成器，以优化模型并实现有效的零样本和少样本分类。