The spread of fake news using out-of-context images has become widespread and
is a challenging task in this era of information overload. Since annotating
huge amounts of such data requires significant time of domain experts, it is
imperative to develop methods which can work in limited annotated data
scenarios. In this work, we explore whether out-of-domain data can help to
improve out-of-context misinformation detection (termed here as multi-modal
fake news detection) of a desired domain, eg. politics, healthcare, etc.
Towards this goal, we propose a novel framework termed DPOD (Domain-specific
Prompt-tuning using Out-of-Domain data). First, to compute generalizable
features, we modify the Vision-Language Model, CLIP to extract features that
helps to align the representations of the images and corresponding text
captions of both the in-domain and out-of-domain data in a label-aware manner.
Further, we propose a domain-specific prompt learning technique which leverages
the training samples of all the available domains based on the the extent they
can be useful to the desired domain. Extensive experiments on a large-scale
benchmark dataset, namely NewsClippings demonstrate that the proposed framework
achieves state of-the-art performance, significantly surpassing the existing
approaches for this challenging task.

本研究旨在探索如何使用领域之外的数据，改进所需领域（如政治、医疗等）的虚假新闻检测，提出了一种名为 DPOD（使用领域特定提示调整的领域特定训练）的新框架，利用图像与文本相应标题的特征对齐，并结合域特定提示学习技术，在大规模基准数据集 NewsClippings 上进行广泛实验，取得了显著超越现有方法的最新性能。