Combating disinformation is one of the burning societal crises -- about 67%
of the American population believes that disinformation produces a lot of
uncertainty, and 10% of them knowingly propagate disinformation. Evidence shows
that disinformation can manipulate democratic processes and public opinion,
causing disruption in the share market, panic and anxiety in society, and even
death during crises. Therefore, disinformation should be identified promptly
and, if possible, mitigated. With approximately 3.2 billion images and 720,000
hours of video shared online daily on social media platforms, scalable
detection of multimodal disinformation requires efficient fact verification.
Despite progress in automatic text-based fact verification (e.g., FEVER, LIAR),
the research community lacks substantial effort in multimodal fact
verification. To address this gap, we introduce FACTIFY 3M, a dataset of 3
million samples that pushes the boundaries of the domain of fact verification
via a multimodal fake news dataset, in addition to offering explainability
through the concept of 5W question-answering. Salient features of the dataset
include: (i) textual claims, (ii) ChatGPT-generated paraphrased claims, (iii)
associated images, (iv) stable diffusion-generated additional images (i.e.,
visual paraphrases), (v) pixel-level image heatmap to foster image-text
explainability of the claim, (vi) 5W QA pairs, and (vii) adversarial fake news
stories.

为解决社交媒体上大规模的多模态虚假信息检测问题，我们介绍了一个名为 FACTIFY 3M 的数据集，包含了 3 百万个样本，其中包括文本声明、图像、像素级图像热力图和 5W 问答对等元素，并且提供了可解释性的概念。