To assist human fact-checkers, researchers have developed automated approaches for visual misinformation detection. These methods assign veracity scores by identifying inconsistencies between the image and its caption, or by detecting forgeries in the image. However, they neglect a crucial point of the human fact-checking process: identifying the original meta-context of the image. By explaining what is actually true about the image, fact-checkers can better detect misinformation, focus their efforts on check-worthy visual content, engage in counter-messaging before misinformation spreads widely, and make their explanation more convincing. Here, we fill this gap by introducing the task of automated image contextualization. We create 5Pils, a dataset of 1,676 fact-checked images with question-answer pairs about their original meta-context. Annotations are based on the 5 Pillars fact-checking framework. We implement a first baseline that grounds the image in its original meta-context using the content of the image and textual evidence retrieved from the open web. Our experiments show promising results while highlighting several open challenges in retrieval and reasoning. We make our code and data publicly available.

本研究针对现有视觉虚假信息检测方法的不足，通过引入自动化图像上下文化任务，填补了识别图像原始元背景的空白。我们创建了5Pils数据集，包含1676张经过事实检查的图像以及其元背景的问答对，实验结果表明该方法具有良好的效果，并指出了在检索和推理中存在的挑战。

“图片，告诉我你的故事！”预测视觉虚假信息的原始元背景