Prompt tuning is a promising method to fine-tune a pre-trained language model
without retraining its large-scale parameters. Instead, it attaches a soft
prompt to the input text, whereby downstream tasks can be well adapted by
merely learning the embeddings of prompt tokens. Nevertheless, existing methods
still suffer from two challenges: (i) they are hard to balance accuracy and
efficiency. A longer (shorter) soft prompt generally leads to a better (worse)
accuracy but at the cost of more (less) training time. (ii) The performance may
not be consistent when adapting to different downstream tasks. We attribute it
to the same embedding space but responsible for different requirements of
downstream tasks. To address these issues, we propose an Efficient Prompt
Tuning method (EPT) by multi-space projection and prompt fusion. Specifically,
it decomposes a given soft prompt into a shorter prompt and two low-rank
matrices, whereby the number of parameters is greatly reduced as well as the
training time. The accuracy is also enhanced by leveraging low-rank matrices
and the short prompt as additional knowledge sources to enrich the semantics of
the original short prompt. In addition, we project the soft prompt into
multiple subspaces to improve the performance consistency, and then adaptively
learn the combination weights of different spaces through a gating network.
Experimental experiments on 13 natural language processing downstream tasks
show that our method significantly and consistently outperforms 11 comparison
methods with the relative percentage of improvements up to 28.8%, and training
time decreased by 14%.

通过多空间投影和提示融合的方式，我们提出了一种高效的提示调整方法（EPT），旨在解决平衡准确性和效率的问题，同时提升语言模型在不同下游任务上的表现。实验结果表明，相对改进百分比高达 28.8％，训练时间减少了 14％。