Vision-language models (VLMs), such as CLIP, have demonstrated impressive
zero-shot capabilities for various downstream tasks. Their performance can be
further enhanced through few-shot prompt tuning methods. However, current
studies evaluate the performance of learned prompts separately on base and new
classes. This evaluation lacks practicality for real-world applications since
downstream tasks cannot determine whether the data belongs to base or new
classes in advance. In this paper, we explore a problem setting called
Open-world Prompt Tuning (OPT), which involves tuning prompts on base classes
and evaluating on a combination of base and new classes. By introducing
Decomposed Prompt Tuning framework (DePT), we theoretically demonstrate that
OPT can be solved by incorporating out-of-distribution detection into prompt
tuning, thereby enhancing the base-to-new discriminability. Based on DePT, we
present a novel prompt tuning approach, namely, Decomposed Context Optimization
(DeCoOp), which introduces new-class detectors and sub-classifiers to further
enhance the base-class and new-class discriminability. Experimental results on
11 benchmark datasets validate the effectiveness of DePT and demonstrate that
DeCoOp outperforms current state-of-the-art methods, providing a significant 2%
average accuracy improvement.

该研究论文介绍了一种名为 Open-world Prompt Tuning (OPT) 的问题设置，通过引入 Decomposed Prompt Tuning framework (DePT) 解决 OPT 问题，并提出了一种名为 Decomposed Context Optimization (DeCoOp) 的新型 prompt tuning 方法，实验证明 DePT 的有效性，并显示 DeCoOp 相较于当前最先进的方法有着 2% 的平均准确率的提升。