Prompt tuning (PT) which only tunes the embeddings of an additional sequence
of tokens per task, keeping the pre-trained language model (PLM) frozen, has
shown remarkable performance in few-shot learning. Despite this, PT has been
shown to rely heavily on good initialization of the prompt embeddings. In this
work, we study meta prompt tuning (MPT) to systematically explore how
meta-learning can help improve (if it can) cross-task generalization in PT
through learning to initialize the prompt embeddings from other relevant tasks.
We empirically analyze a representative set of meta learning algorithms in a
wide range of adaptation settings with different source/target task
configurations on a large set of few-shot tasks. With extensive experiments and
analysis, we demonstrate the effectiveness of MPT. We find the improvement to
be significant particularly on classification tasks. For other kinds of tasks
such as question answering, we observe that while MPT can outperform PT in most
cases, it does not always outperform multi-task learning. We further provide an
in-depth analysis from the perspective of task similarity.

本文研究了元提示调整（meta prompt tuning）来探索元学习如何通过学习从其他相关任务中初始化提示嵌入来帮助改善 PT 中的跨任务推广。我们在广泛的适应设置中经验性地分析了代表性的元学习算法，并从任务相似性的角度提供了深入的分析。实验结果表明，MPT 的改进在分类任务中显著。