This study investigates the integration of diverse patient data sources into
multimodal language models for automated chest X-ray (CXR) report generation.
Traditionally, CXR report generation relies solely on CXR images and limited
radiology data, overlooking valuable information from patient health records,
particularly from emergency departments. Utilising the MIMIC-CXR and
MIMIC-IV-ED datasets, we incorporate detailed patient information such as
aperiodic vital signs, medications, and clinical history to enhance diagnostic
accuracy. We introduce a novel approach to transform these heterogeneous data
sources into embeddings that prompt a multimodal language model, significantly
enhancing the diagnostic accuracy of generated radiology reports. Our
comprehensive evaluation demonstrates the benefits of using a broader set of
patient data, underscoring the potential for enhanced diagnostic capabilities
and better patient outcomes through the integration of multimodal data in CXR
report generation.

该研究调查了将多样的患者数据源整合到多模态语言模型中，用于自动化胸部 X 射线报告生成。通过使用 MIMIC-CXR 和 MIMIC-IV-ED 数据集，该研究通过整合详细的患者信息，如不定期的生命体征、药物和临床病史，将异质数据源转化为嵌入向量，从而显著提高了生成放射学报告的诊断准确性。全面的评估显示了使用更广泛的患者数据的好处，强调了通过将多模态数据整合到胸部 X 射线报告生成中可以提高诊断能力和改善患者预后的潜力。