Zero-shot inference, where pre-trained models perform tasks without specific training data, is an exciting emergent ability of large models like CLIP. Although there has been considerable exploration into enhancing zero-shot abilities in image captioning (IC) for popular datasets such as MSCOCO and Flickr8k, these approaches fall short with fine-grained datasets like CUB, FLO, UCM-Captions, and Sydney-Captions. These datasets require captions to discern between visually and semantically similar classes, focusing on detailed object parts and their attributes. To overcome this challenge, we introduce TRaining-Free Object-Part Enhancement (TROPE). TROPE enriches a base caption with additional object-part details using object detector proposals and Natural Language Processing techniques. It complements rather than alters the base caption, allowing seamless integration with other captioning methods and offering users enhanced flexibility. Our evaluations show that TROPE consistently boosts performance across all tested zero-shot IC approaches and achieves state-of-the-art results on fine-grained IC datasets.

本研究解决了细粒度数据集在零-shot图像描述中存在的不足，尤其是在细微对象部分与属性的区分能力不足的问题。通过引入无训练对象部分增强技术（TROPE），精细化描述了对象部分细节，提升了描述质量。研究结果表明，TROPE显著提高了各类零-shot图像描述方法的表现，并在细粒度数据集上达到了最先进的成果。

无训练对象部分增强技术：无缝提升细粒度零样本图像描述