Recently, prompt learning has become a new paradigm to utilize pre-trained
language models (PLMs) and achieves promising results in downstream tasks with
a negligible increase of parameters. The current usage of discrete and
continuous prompts assumes that the prompt is fixed for a specific task and all
samples in the task share the same prompt. However, a task may contain quite
diverse samples in which some are easy and others are difficult, and diverse
prompts are desirable. In this paper, we propose an instance-aware prompt
learning method that learns a different prompt for each instance. Specifically,
we suppose that each learnable prompt token has a different contribution to
different instances, and we learn the contribution by calculating the relevance
score between an instance and each prompt token. The contribution weighted
prompt would be instance aware. We apply our method to both unidirectional and
bidirectional PLMs on both language understanding and generation tasks.
Extensive experiments demonstrate that our method obtains considerable
improvements compared to strong baselines. Especially, our method achieves the
state-of-the-art on the SuperGLUE few-shot learning benchmark.

本文提出了一种基于实例的提示学习方法，用于不同实例的学习。该方法在双向和单向的 PLMs 上得到了相当大的提升，并在 SuperGLUE few-shot learning 基准测试上实现了最好的结果。