This report introduces a solution to the Topic 1 Zero-shot Image Captioning
of 2024 NICE : New frontiers for zero-shot Image Captioning Evaluation. In
contrast to NICE 2023 datasets, this challenge involves new annotations by
humans with significant differences in caption style and content. Therefore, we
enhance image captions effectively through retrieval augmentation and caption
grading methods. At the data level, we utilize high-quality captions generated
by image caption models as training data to address the gap in text styles. At
the model level, we employ OFA (a large-scale visual-language pre-training
model based on handcrafted templates) to perform the image captioning task.
Subsequently, we propose caption-level strategy for the high-quality caption
data generated by the image caption models and integrate them with retrieval
augmentation strategy into the template to compel the model to generate higher
quality, more matching, and semantically enriched captions based on the
retrieval augmentation prompts. Our approach ranks first on the leaderboard,
achieving a CIDEr score of 234.11 and 1st in all other metrics.

本文介绍了一种解决 2024 年 NICE 的 Topic 1 零射击图像字幕挑战的方法，通过检索增强和字幕评分方法有效地增强图像字幕，利用图像字幕模型生成的高质量字幕作为训练数据，利用基于手工模板的大规模视觉 - 语言预训练模型（OFA）执行图像字幕任务，并将字幕级策略与检索增强策略相结合，以生成更高质量、更匹配和语义丰富的字幕。基于这种方法，在排行榜上排名第一，在所有其他指标上都取得了第一名的成绩。