Out-of-context (OOC) detection is a challenging task involving identifying
images and texts that are irrelevant to the context in which they are
presented. Large vision-language models (LVLMs) are effective at various tasks,
including image classification and text generation. However, the extent of
their proficiency in multimodal OOC detection tasks is unclear. In this paper,
we investigate the ability of LVLMs to detect multimodal OOC and show that
these models cannot achieve high accuracy on OOC detection tasks without
fine-tuning. However, we demonstrate that fine-tuning LVLMs on multimodal OOC
datasets can further improve their OOC detection accuracy. To evaluate the
performance of LVLMs on OOC detection tasks, we fine-tune MiniGPT-4 on the
NewsCLIPpings dataset, a large dataset of multimodal OOC. Our results show that
fine-tuning MiniGPT-4 on the NewsCLIPpings dataset significantly improves the
OOC detection accuracy in this dataset. This suggests that fine-tuning can
significantly improve the performance of LVLMs on OOC detection tasks.

证明了在多模态的背景下，通过对数据集进行微调，可以显著提高大视觉 - 语言模型在超文本检测任务中的性能。