Despite the success of Large Language Models (LLMs) in general image tasks, a
gap persists in the medical field for a multimodal large model adept at
handling the nuanced diversity of medical images. Addressing this, we propose
MedXChat, a unified multimodal large model designed for seamless interactions
between medical assistants and users. MedXChat encompasses three key
functionalities: CXR(Chest X-ray)-to-Report generation, CXR-based visual
question-answering (VQA), and Text-to-CXR synthesis. Our contributions are as
follows. Firstly, our model showcases exceptional cross-task adaptability,
displaying adeptness across all three defined tasks and outperforming the
benchmark models on the MIMIC dataset in medical multimodal applications.
Secondly, we introduce an innovative Text-to-CXR synthesis approach that
utilizes instruction-following capabilities within the Stable Diffusion (SD)
architecture. This technique integrates smoothly with the existing model
framework, requiring no extra parameters, thereby maintaining the SD's
generative strength while also bestowing upon it the capacity to render
fine-grained medical images with high fidelity. Comprehensive experiments
validate MedXChat's synergistic enhancement across all tasks. Our instruction
data and model will be open-sourced.

MedXChat 是一个用于医学助理和用户之间无缝互动的统一多模态大型模型，包括 CXR 到报告生成、基于 CXR 的视觉问答和文本到 CXR 合成三个关键功能。该模型在医学多模态应用中显示出优异的跨任务适应性，并在 MIMIC 数据集上的性能超越了基准模型。此外，该研究还介绍了一种创新的文本到 CXR 合成方法，利用了 Stable Diffusion（SD）架构内的指令跟随能力，无需额外参数，使模型能够生成高保真度的精细化医学图像。详尽的实验证实了 MedXChat 在所有任务上的协同增强效果。研究中的指令数据和模型将开源。