Recent progress has shown great potential of visual prompt tuning (VPT) when adapting pre-trained vision transformers to various downstream tasks. However, most existing solutions independently optimize prompts at each layer, thereby neglecting the usage of task-relevant information encoded in prompt tokens across layers. Additionally, existing prompt structures are prone to interference from task-irrelevant noise in input images, which can do harm to the sharing of task-relevant information. In this paper, we propose a novel VPT approach, \textbf{iVPT}. It innovatively incorporates a cross-layer dynamic connection (CDC) for input prompt tokens from adjacent layers, enabling effective sharing of task-relevant information. Furthermore, we design a dynamic aggregation (DA) module that facilitates selective sharing of information between layers. The combination of CDC and DA enhances the flexibility of the attention process within the VPT framework. Building upon these foundations, iVPT introduces an attentive reinforcement (AR) mechanism, by automatically identifying salient image tokens, which are further enhanced by prompt tokens in an additive manner. Extensive experiments on 24 image classification and semantic segmentation benchmarks clearly demonstrate the advantage of the proposed iVPT, compared to the state-of-the-art counterparts.

最近的进展显示出了将预训练的视觉转换器应用到各种下游任务中的视觉提示调整（VPT）的巨大潜力。本文提出了一种创新的VPT方法，即iVPT，它通过从相邻层的输入提示令牌中引入跨层动态连接（CDC），实现了任务相关信息的有效共享，并采用动态聚合（DA）模块促进了层间信息的选择性共享。基于这些基础，iVPT引入了一种关注强化（AR）机制，通过自动识别显著的图像令牌，并以增加的方式与提示令牌进一步增强。对24个图像分类和语义分割基准的大量实验证明了所提出的iVPT相对于现有最先进方法的优势。

iVPT：通过跨层动态连接改进视觉提示调整中的任务相关信息共享