Recent advances for few-shot text classification aim to wrap textual inputs
with task-specific prompts to cloze questions. By processing them with a masked
language model to predict the masked tokens and using a verbalizer that
constructs the mapping between predicted words and target labels. This approach
of using pre-trained language models is called prompt-based tuning, which could
remarkably outperform conventional fine-tuning approach in the low-data
scenario. As the core of prompt-based tuning, the verbalizer is usually
handcrafted with human efforts or suboptimally searched by gradient descent. In
this paper, we focus on automatically constructing the optimal verbalizer and
propose a novel evolutionary verbalizer search (EVS) algorithm, to improve
prompt-based tuning with the high-performance verbalizer. Specifically,
inspired by evolutionary algorithm (EA), we utilize it to automatically evolve
various verbalizers during the evolutionary procedure and select the best one
after several iterations. Extensive few-shot experiments on five text
classification datasets show the effectiveness of our method.

本文提出了一种新的进化式口头表述搜索算法用于构建最优的口头表述器，在少量数据情况下使用预训练的语言模型以及任务特定提示能够极大地提高准确性，进一步推进了小样本文本分类的研究。

基于提示的少样本文本分类的进化语言生成搜索

Evolutionary Verbalizer Search for Prompt-based Few Shot Text  Classification

We study the problem of few-shot Fine-grained Entity Typing (FET), where only
a few annotated entity mentions with contexts are given for each entity type.
Recently, prompt-based tuning has demonstrated superior performance to standard
fine-tuning in few-shot scenarios by formulating the entity type classification
task as a ''fill-in-the-blank'' problem. This allows effective utilization of
the strong language modeling capability of Pre-trained Language Models (PLMs).
Despite the success of current prompt-based tuning approaches, two major
challenges remain: (1) the verbalizer in prompts is either manually designed or
constructed from external knowledge bases, without considering the target
corpus and label hierarchy information, and (2) current approaches mainly
utilize the representation power of PLMs, but have not explored their
generation power acquired through extensive general-domain pre-training. In
this work, we propose a novel framework for few-shot FET consisting of two
modules: (1) an entity type label interpretation module automatically learns to
relate type labels to the vocabulary by jointly leveraging few-shot instances
and the label hierarchy, and (2) a type-based contextualized instance generator
produces new instances based on given instances to enlarge the training set for
better generalization. On three benchmark datasets, our model outperforms
existing methods by significant margins. Code can be found at
this https URL.

本文提出了一个新的架构，包括两个模块：（1）实体类型标签解释模块，自动学习将类型标签与词汇表相关联的方法，（2）基于类型的上下文实例生成器，根据给定实例生成新实例，以扩大训练集以获得更好的泛化能力。在三个基准数据集上，该模型的性能明显高于现有方法。

自动标签解释与实例生成的少样本细粒度实体类型划分

Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation

Recent advances in large pre-trained language models (PLMs) lead to
impressive gains in natural language understanding (NLU) tasks with
task-specific fine-tuning. However, directly fine-tuning PLMs heavily relies on
sufficient labeled training instances, which are usually hard to obtain.
Prompt-based tuning on PLMs has shown to be powerful for various downstream
few-shot tasks. Existing works studying prompt-based tuning for few-shot NLU
tasks mainly focus on deriving proper label words with a verbalizer or
generating prompt templates to elicit semantics from PLMs. In addition,
conventional data augmentation strategies such as synonym substitution, though
widely adopted in low-resource scenarios, only bring marginal improvements for
prompt-based few-shot learning. Thus, an important research question arises:
how to design effective data augmentation methods for prompt-based few-shot
tuning? To this end, considering the label semantics are essential in
prompt-based tuning, we propose a novel label-guided data augmentation
framework PromptDA, which exploits the enriched label semantic information for
data augmentation. Extensive experiment results on few-shot text classification
tasks demonstrate the superior performance of the proposed framework by
effectively leveraging label semantics and data augmentation for natural
language understanding. Our code is available at
this https URL.

本研究利用丰富的标签语义信息提出了一种新颖的标签引导数据增强框架 PromptDA，通过有效地利用标签语义和数据增强来提高自然语言理解的性能，以解决如何为基于 Prompt 的 few-shot 调整设计有效的数据增强方法的重要研究问题。