Recent progress in deterministic prompt learning has become a promising
alternative to various downstream vision tasks, enabling models to learn
powerful visual representations with the help of pre-trained vision-language
models. However, this approach results in limited performance for dense
prediction tasks that require handling more complex and diverse objects, since
a single and deterministic description cannot sufficiently represent the entire
image. In this paper, we present a novel probabilistic prompt learning to fully
exploit the vision-language knowledge in dense prediction tasks. First, we
introduce learnable class-agnostic attribute prompts to describe universal
attributes across the object class. The attributes are combined with class
information and visual-context knowledge to define the class-specific textual
distribution. Text representations are sampled and used to guide the dense
prediction task using the probabilistic pixel-text matching loss, enhancing the
stability and generalization capability of the proposed method. Extensive
experiments on different dense prediction tasks and ablation studies
demonstrate the effectiveness of our proposed method.

本文介绍了一种新的概率提示学习方法，利用可学习的类别不可知属性提示描述对象类的通用属性，并与类别信息和视觉上下文知识相结合，定义类别特定的文本分布，通过使用概率像素文本匹配损失来指导密集预测任务，增强了所提出方法的稳定性和泛化能力。实验结果表明了我们提出的方法的有效性。