Parameter efficient transfer learning (PETL) is an emerging research spot
that aims to adapt large-scale pre-trained models to downstream tasks. Recent
advances have achieved great success in saving storage and computation costs.
However, these methods do not take into account instance-specific visual clues
for visual tasks. In this paper, we propose a Dynamic Visual Prompt Tuning
framework (DVPT), which can generate a dynamic instance-wise token for each
image. In this way, it can capture the unique visual feature of each image,
which can be more suitable for downstream visual tasks. We designed a Meta-Net
module that can generate learnable prompts based on each image, thereby
capturing dynamic instance-wise visual features. Extensive experiments on a
wide range of downstream recognition tasks show that DVPT achieves superior
performance than other PETL methods. More importantly, DVPT even outperforms
full fine-tuning on 17 out of 19 downstream tasks while maintaining high
parameter efficiency. Our code will be released soon.

参数有效的迁移学习 (PETL) 是一个新兴的研究领域，旨在将大规模预训练模型适应下游任务。我们提出了一个动态视觉提示调整框架 (DVPT)，可以为每个图像生成动态的实例级标记，以捕捉每个图像的独特视觉特征，从而更适合下游视觉任务。通过对广泛的下游识别任务进行实验，我们发现 DVPT 方法在性能上优于其他 PETL 方法，甚至在 19 个下游任务中有 17 个超越了完全微调的性能，同时保持了高参数效率。