This paper proposes a novel framework for multi-label image recognition
without any training data, called data-free framework, which uses knowledge of
pre-trained Large Language Model (LLM) to learn prompts to adapt pretrained
Vision-Language Model (VLM) like CLIP to multilabel classification. Through
asking LLM by well-designed questions, we acquire comprehensive knowledge about
characteristics and contexts of objects, which provides valuable text
descriptions for learning prompts. Then we propose a hierarchical prompt
learning method by taking the multi-label dependency into consideration,
wherein a subset of category-specific prompt tokens are shared when the
corresponding objects exhibit similar attributes or are more likely to
co-occur. Benefiting from the remarkable alignment between visual and
linguistic semantics of CLIP, the hierarchical prompts learned from text
descriptions are applied to perform classification of images during inference.
Our framework presents a new way to explore the synergies between multiple
pre-trained models for novel category recognition. Extensive experiments on
three public datasets (MS-COCO, VOC2007, and NUS-WIDE) demonstrate that our
method achieves better results than the state-of-the-art methods, especially
outperforming the zero-shot multi-label recognition methods by 4.7% in mAP on
MS-COCO.

提出了一种无需训练数据的多标签图像识别新框架，利用预训练大型语言模型（LLM）的知识学习提示，使预训练的视觉 - 语言模型（VLM）如 CLIP 适应多标签分类。通过向 LLM 提问获取关于对象的特性和背景的综合知识，为学习提示提供宝贵的文本描述。然后，通过考虑多标签依赖性，提出了一种层次化的提示学习方法，在对象具有相似属性或更有可能共现时，共享特定类别提示标记的子集。由于 CLIP 在视觉和语义上具有显著的对准性，从文本描述学习到的层次化提示被应用于推理过程中的图像分类。我们的框架为探索多个预训练模型之间的协同作用提供了一种新途径。在三个公共数据集（MS-COCO、VOC2007 和 NUS-WIDE）上进行的大量实验证明，我们的方法比现有方法取得更好的结果，尤其是在 MS-COCO 上的零样本多标签识别方法的 mAP 上超过 4.7%。