The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has
enhanced medical diagnosis. However, current Med-LVLMs frequently encounter
factual issues, often generating responses that do not align with established
medical facts. Retrieval-Augmented Generation (RAG), which utilizes external
knowledge, can improve the factual accuracy of these models but introduces two
major challenges. First, limited retrieved contexts might not cover all
necessary information, while excessive retrieval can introduce irrelevant and
inaccurate references, interfering with the model's generation. Second, in
cases where the model originally responds correctly, applying RAG can lead to
an over-reliance on retrieved contexts, resulting in incorrect answers. To
address these issues, we propose RULE, which consists of two components. First,
we introduce a provably effective strategy for controlling factuality risk
through the calibrated selection of the number of retrieved contexts. Second,
based on samples where over-reliance on retrieved contexts led to errors, we
curate a preference dataset to fine-tune the model, balancing its dependence on
inherent knowledge and retrieved contexts for generation. We demonstrate the
effectiveness of RULE on three medical VQA datasets, achieving an average
improvement of 20.8% in factual accuracy. We publicly release our benchmark and
code in this https URL

基于外部知识的检索增强生成（RAG）方法，通过控制检索上下文数量和精心策划首选数据集来提高医学大规模视觉语言模型（Med-LVLMs）的事实准确性，并在三个医学 VQA 数据集上实现了平均 20.8% 的改进。

可靠的多模式医学视觉语言模型中的事实性规则

RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language  Models

Artificial intelligence has significantly impacted medical applications,
particularly with the advent of Medical Large Vision Language Models
(Med-LVLMs), sparking optimism for the future of automated and personalized
healthcare. However, the trustworthiness of Med-LVLMs remains unverified,
posing significant risks for future model deployment. In this paper, we
introduce CARES and aim to comprehensively evaluate the Trustworthiness of
Med-LVLMs across the medical domain. We assess the trustworthiness of Med-LVLMs
across five dimensions, including trustfulness, fairness, safety, privacy, and
robustness. CARES comprises about 41K question-answer pairs in both closed and
open-ended formats, covering 16 medical image modalities and 27 anatomical
regions. Our analysis reveals that the models consistently exhibit concerns
regarding trustworthiness, often displaying factual inaccuracies and failing to
maintain fairness across different demographic groups. Furthermore, they are
vulnerable to attacks and demonstrate a lack of privacy awareness. We publicly
release our benchmark and code in this https URL

通过 CARES 综合评估医学领域中基于 Med-LVLMs 的可信度，并发现这些模型在可信度、公平性、安全性、隐私保护和稳健性等方面存在问题。