Prompt tuning, or the conditioning of a frozen pretrained language model (PLM) with soft prompts learned from data, has demonstrated impressive performance on a wide range of NLP tasks. However, prompt tuning requires a large training dataset to be effective and is outperformed by finetuning the entire PLM in data-scarce regimes. Previous work \citep{gu-etal-2022-ppt,vu-etal-2022-spot} proposed to transfer soft prompts pretrained on the source domain to the target domain. In this paper, we explore domain adaptation for prompt tuning, a problem setting where unlabeled data from the target domain are available during pretraining. We propose bOosting Prompt TunIng with doMain Adaptation (OPTIMA), which regularizes the decision boundary to be smooth around regions where source and target data distributions are similar. Extensive experiments demonstrate that OPTIMA significantly enhances the transferability and sample-efficiency of prompt tuning compared to strong baselines. Moreover, in few-shot settings, OPTIMA exceeds full-model tuning by a large margin.

本文提出了一种名为OPTIMA的算法，通过领域自适应来改进预处理语言模型的prompt tuning，结果表明OPTIMA可以显著提高prompt tuning的可迁移性和样本效率，并在少样本情况下超过全模型调整性能。

通过领域适应改善Prompt Tuning的样本效率