Prompt tuning, in which a base pretrained model is adapted to each task via
conditioning on learned prompt vectors, has emerged as a promising approach for
efficiently adapting large language models to multiple downstream tasks.
However, existing methods typically learn soft prompt vectors from scratch, and
it has not been clear how to exploit the rich cross-task knowledge with prompt
vectors in a multitask learning setting. We propose multitask prompt tuning
(MPT), which first learns a single transferable prompt by distilling knowledge
from multiple task-specific source prompts. We then learn multiplicative low
rank updates to this shared prompt to efficiently adapt it to each downstream
target task. Extensive experiments on 23 NLP datasets demonstrate that our
proposed approach outperforms the state-of-the-art methods, including the full
finetuning baseline in some cases, despite only tuning 0.035% as many
task-specific parameters.

我们提出了多任务提示调整（MPT）方法，该方法通过从多个特定于任务的源提示中提取知识来首先学习一个单一可传输的提示，然后学习该共享提示的乘性低秩更新，以高效地将其适应每个下游目标任务。在 23 个自然语言处理数据集上进行的广泛实验表明，我们的方法在某些情况下优于最先进的方法，包括完全微调基线方法，尽管只有 0.035％的特定于任务的参数被调整。