Humans possess the capacity to reason about the future based on a sparse
collection of visual cues acquired over time. In order to emulate this ability,
we introduce a novel task called Anticipation Captioning, which generates a
caption for an unseen oracle image using a sparsely temporally-ordered set of
images. To tackle this new task, we propose a model called A-CAP, which
incorporates commonsense knowledge into a pre-trained vision-language model,
allowing it to anticipate the caption. Through both qualitative and
quantitative evaluations on a customized visual storytelling dataset, A-CAP
outperforms other image captioning methods and establishes a strong baseline
for anticipation captioning. We also address the challenges inherent in this
task.

通过引入一种新任务，即预期字幕，来模拟人类基于时间顺序的视觉暗示进行未来思考的能力，并提出了一种名为 A-CAP 的模型，该模型将常识知识并入预训练的视觉和语言模型中，使其能够进行预期字幕，并在定制的视觉叙述数据集上进行定量和定性评估，A-CAP 优于其他图像字幕方法，为预期字幕建立了一个强大的基线，同时解决了该任务的内在挑战。