Pre-trained Vision-Language (V-L) models set the benchmark for generalization
to downstream tasks among the noteworthy contenders. Many characteristics of
the V-L model have been explored in existing research including the challenge
of the sensitivity to text input and the tuning process across multi-modal
prompts. With the advanced utilization of the V-L model like CLIP, recent
approaches deploy learnable prompts instead of hand-craft prompts to boost the
generalization performance and address the aforementioned challenges. Inspired
by layer-wise training, which is wildly used in image fusion, we note that
using a sequential training process to adapt different modalities branches of
CLIP efficiently facilitates the improvement of generalization. In the context
of addressing the multi-modal prompting challenge, we propose Token-wise
Adaptive for Multi-modal Prompt Learning (APLe) for tuning both modalities
prompts, vision and language, as tokens in a sequential manner. APLe addresses
the challenges in V-L models to promote prompt learning across both modalities,
which indicates a competitive generalization performance in line with the
state-of-the-art. Preeminently, APLe shows robustness and favourable
performance in prompt-length experiments with an absolute advantage in adopting
the V-L models.

通过使用 Token-wise Adaptive for Multi-modal Prompt Learning (APLe) 在顺序方式中对视觉和语言两个模态的提示进行调整，APLe 解决了视觉 - 语言模型中的挑战，提高了提示学习的性能，具有与最先进技术相媲美的泛化性能。