Pre-trained vision-language models have notably accelerated progress of
open-world concept recognition. Their impressive zero-shot ability has recently
been transferred to multi-label image classification via prompt tuning,
enabling to discover novel labels in an open-vocabulary manner. However, this
paradigm suffers from non-trivial training costs, and becomes computationally
prohibitive for a large number of candidate labels. To address this issue, we
note that vision-language pre-training aligns images and texts in a unified
embedding space, making it potential for an adapter network to identify labels
in visual modality while be trained in text modality. To enhance such
cross-modal transfer ability, a simple yet effective method termed random
perturbation is proposed, which enables the adapter to search for potential
visual embeddings by perturbing text embeddings with noise during training,
resulting in better performance in visual modality. Furthermore, we introduce
an effective approach to employ large language models for multi-label
instruction-following text generation. In this way, a fully automated pipeline
for visual label recognition is developed without relying on any manual data.
Extensive experiments on public benchmarks show the superiority of our method
in various multi-label classification tasks.

通过预先训练的视觉 - 语言模型，结合适当的调整和随机扰动方法构建了一个能够自动识别图像标签的全自动流水线，该方法在各种多标签分类任务中显示出优越性能。