In prompt tuning, a prefix or suffix text is added to the prompt, and the
embeddings (soft prompts) or token indices (hard prompts) of the prefix/suffix
are optimized to gain more control over language models for specific tasks.
This approach eliminates the need for hand-crafted prompt engineering or
explicit model fine-tuning. Prompt tuning is significantly more
parameter-efficient than model fine-tuning, as it involves optimizing partial
inputs of language models to produce desired outputs.
In this work, we aim to further reduce the amount of trainable parameters
required for a language model to perform well on specific tasks. We propose
Low-rank Prompt Tuning (LoPT), a low-rank model for prompts that achieves
efficient prompt optimization. The proposed method demonstrates similar
outcomes to full parameter prompt tuning while reducing the number of trainable
parameters by a factor of 5. It also provides promising results compared to the
state-of-the-art methods that would require 10 to 20 times more parameters.

在本论文中，我们提出了一种名为 LoPT 的低秩模型，用于优化提示信息，通过减少可训练参数数量，实现与完全参数的提示优化相似的效果，同时相较于现有方法需要的参数数量减少了 5 到 20 倍。