Prompt ensembling of Large Language Model (LLM) generated category-specific
prompts has emerged as an effective method to enhance zero-shot recognition
ability of Vision-Language Models (VLMs). To obtain these category-specific
prompts, the present methods rely on hand-crafting the prompts to the LLMs for
generating VLM prompts for the downstream tasks. However, this requires
manually composing these task-specific prompts and still, they might not cover
the diverse set of visual concepts and task-specific styles associated with the
categories of interest. To effectively take humans out of the loop and
completely automate the prompt generation process for zero-shot recognition, we
propose Meta-Prompting for Visual Recognition (MPVR). Taking as input only
minimal information about the target task, in the form of its short natural
language description, and a list of associated class labels, MPVR automatically
produces a diverse set of category-specific prompts resulting in a strong
zero-shot classifier. MPVR generalizes effectively across various popular
zero-shot image recognition benchmarks belonging to widely different domains
when tested with multiple LLMs and VLMs. For example, MPVR obtains a zero-shot
recognition improvement over CLIP by up to 19.8% and 18.2% (5.0% and 4.5% on
average over 20 datasets) leveraging GPT and Mixtral LLMs, respectively

通过 Meta-Prompting for Visual Recognition (MPVR) 方法，仅凭目标任务的简短自然语言描述和相关类别标签的最小信息输入，自动产生一组多样的类别特定提示，从而实现强大的零样本分类器。在多个不同领域的流行的零样本图像识别基准上，使用多个 LLMs 和 VLMs 测试，MPVR 可以有效地推广，比 CLIP 提高了最高 19.8％和 18.2％（平均 20 个数据集上分别为 5.0％和 4.5％），依赖于 GPT 和 Mixtral LLMs。