This work breaks through the Base-New Tradeoff (BNT)dilemma in prompt tuning,
i.e., the better the tuned model generalizes to the base (or target) task, the
worse it generalizes to new tasks, and vice versa. Specifically, through an
in-depth analysis of the learned features of the base and new tasks, we observe
that the BNT stems from a channel bias issue, i.e., the vast majority of
feature channels are occupied by base-specific knowledge, resulting in the
collapse of taskshared knowledge important to new tasks. To address this, we
propose the Decoupled Prompt Tuning (DePT) framework, which decouples
base-specific knowledge from feature channels into an isolated feature space
during prompt tuning, so as to maximally preserve task-shared knowledge in the
original feature space for achieving better zero-shot generalization on new
tasks. Importantly, our DePT is orthogonal to existing prompt tuning methods,
hence it can improve all of them. Extensive experiments on 11 datasets show the
strong flexibility and effectiveness of DePT. Our code and pretrained models
are available at this https URL

该研究通过解决提示调整中的基础 - 新任务权衡困境，提出了一种基于 Decoupled Prompt Tuning (DePT) 框架的方法，通过将基础特定知识与特征通道解耦，最大程度地保留原特征空间中的任务共享知识，从而在新任务中实现更好的零样本泛化能力。