Multimodal large language models (MLLMs) carry the potential to support
humans in processing vast amounts of information. While MLLMs are already being
used as a fact-checking tool, their abilities and limitations in this regard
are understudied. Here is aim to bridge this gap. In particular, we propose a
framework for systematically assessing the capacity of current multimodal
models to facilitate real-world fact-checking. Our methodology is
evidence-free, leveraging only these models' intrinsic knowledge and reasoning
capabilities. By designing prompts that extract models' predictions,
explanations, and confidence levels, we delve into research questions
concerning model accuracy, robustness, and reasons for failure. We empirically
find that (1) GPT-4V exhibits superior performance in identifying malicious and
misleading multimodal claims, with the ability to explain the unreasonable
aspects and underlying motives, and (2) existing open-source models exhibit
strong biases and are highly sensitive to the prompt. Our study offers insights
into combating false multimodal information and building secure, trustworthy
multimodal models. To the best of our knowledge, we are the first to evaluate
MLLMs for real-world fact-checking.

对多模态大型语言模型在事实检查方面的能力和局限性进行了系统评估，发现 GPT-4V 在识别恶意和误导性多模态论断方面表现出优越性能，具备解释不合理方面和潜在动机的能力，同时已有的开源模型存在强烈的偏见，并对提示非常敏感。这项研究为对抗虚假多模态信息和构建安全可靠的多模态模型提供了启示，据我们所知，这是第一次对多模态大型语言模型进行真实世界事实检查的评估。

支持现实世界事实核查的多模态大型语言模型

Multimodal Large Language Models to Support Real-World Fact-Checking

This article introduces analytical techniques and a decision support tool to
support capacity assessment and case mix planning (CMP) approaches previously
created for hospitals. First, an optimization model is proposed to analyse the
impact of making a change to an existing case mix. This model identifies how
other patient types should be altered proportionately to the changing levels of
hospital resource availability. Then we propose multi-objective decision-making
techniques to compare and critique competing case mix solutions obtained. The
proposed techniques are embedded seamlessly within an Excel Visual Basic for
Applications (VBA) personal decision support tool (PDST), for performing
informative quantitative assessments of hospital capacity. The PDST reports
informative metrics of difference and reports the impact of case mix
modifications on the other types of patient present. The techniques developed
in this article provide a bridge between theory and practice that is currently
missing and provides further situational awareness around hospital capacity.

该文章介绍了分析技术和决策支持工具，用于支持之前为医院创建的容量评估和病例组合规划（CMP）方法。提出了一个优化模型来分析对现有病例组合进行更改的影响，并确定其他类型的患者应该以何种比例随着医院资源的变化而进行相应的改变。然后，我们提出了多目标决策技术，用于比较和评价获得的竞争性病例组合解决方案。提出的技术无缝嵌入了一个 Excel 视觉基础应用程序（VBA）个人决策支持工具（PDST），用于对医院容量进行信息量化评估。PDST 报告差异的信息度量，并报告病例组合修改对其他类型患者的影响。本文所开发的技术弥补了当前理论与实践之间的缺失，并提供了更多有关医院容量的情境意识。