Existing claim verification datasets often do not require systems to perform complex reasoning or effectively interpret multimodal evidence. To address this, we introduce a new task: multi-hop multimodal claim verification. This task challenges models to reason over multiple pieces of evidence from diverse sources, including text, images, and tables, and determine whether the combined multimodal evidence supports or refutes a given claim. To study this task, we construct MMCV, a large-scale dataset comprising 16k multi-hop claims paired with multimodal evidence, generated and refined using large language models, with additional input from human feedback. We show that MMCV is challenging even for the latest state-of-the-art multimodal large language models, especially as the number of reasoning hops increases. Additionally, we establish a human performance benchmark on a subset of MMCV. We hope this dataset and its evaluation task will encourage future research in multimodal multi-hop claim verification.

现有的声明验证数据集通常不要求系统进行复杂推理或有效解释多模态证据。为了解决这一问题，我们提出了一项新任务：多跳多模态声明验证。研究发现，MMC数据集在多跳推理的挑战性上对最新的多模态大语言模型构成了威胁，期待此数据集能推动未来在多模态多跳声明验证领域的研究。

整合所有信息：验证多跳多模态声明