Image captioning models are widely used to describe recent and archived
pictures with the objective of improving their accessibility and retrieval.
Yet, these approaches tend to be inefficient and biased at retrieving people's
names. In this work we introduce AstroCaptions, a dataset for the image
captioning task. This dataset specifically contains thousands of public
fig-ures that are complex to identify for a traditional model. We also propose
a novel post-processing method to insert identified people's names inside the
caption using explainable AI tools and the grounding capabilities of
vi-sion-language models. The results obtained with this method show
signifi-cant improvements of captions quality and a potential of reducing
halluci-nations. Up to 93.2% of the persons detected can be inserted in the
image captions leading to improvements in the BLEU, ROUGE, CIDEr and METEOR
scores of each captioning model.

介绍了 AstroCaptions 数据集，该数据集用于图像描述任务，包含了许多公众人物图像，同时使用可解释的人工智能工具和视觉语言模型的定位能力，提出了一种将识别出的人物姓名插入标题的新方法，取得了显著的描述质量提升和减少幻觉的潜力。