This study investigates the integration of diverse patient data sources into
multimodal language models for automated chest X-ray (CXR) report generation.
Traditionally, CXR report generation relies solely on CXR images and limited
radiology data, overlooking valuable information from patient health records,
particularly from emergency departments. Utilising the MIMIC-CXR and
MIMIC-IV-ED datasets, we incorporate detailed patient information such as
aperiodic vital signs, medications, and clinical history to enhance diagnostic
accuracy. We introduce a novel approach to transform these heterogeneous data
sources into embeddings that prompt a multimodal language model, significantly
enhancing the diagnostic accuracy of generated radiology reports. Our
comprehensive evaluation demonstrates the benefits of using a broader set of
patient data, underscoring the potential for enhanced diagnostic capabilities
and better patient outcomes through the integration of multimodal data in CXR
report generation.

该研究调查了将多样的患者数据源整合到多模态语言模型中，用于自动化胸部 X 射线报告生成。通过使用 MIMIC-CXR 和 MIMIC-IV-ED 数据集，该研究通过整合详细的患者信息，如不定期的生命体征、药物和临床病史，将异质数据源转化为嵌入向量，从而显著提高了生成放射学报告的诊断准确性。全面的评估显示了使用更广泛的患者数据的好处，强调了通过将多模态数据整合到胸部 X 射线报告生成中可以提高诊断能力和改善患者预后的潜力。

辅助患者数据对自动胸部 X 射线报告生成的影响及如何整合

The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report  Generation and How to Incorporate It

Intensive Care Units (ICU) require comprehensive patient data integration for
enhanced clinical outcome predictions, crucial for assessing patient
conditions. Recent deep learning advances have utilized patient time series
data, and fusion models have incorporated unstructured clinical reports,
improving predictive performance. However, integrating established medical
knowledge into these models has not yet been explored. The medical domain's
data, rich in structural relationships, can be harnessed through knowledge
graphs derived from clinical ontologies like the Unified Medical Language
System (UMLS) for better predictions. Our proposed methodology integrates this
knowledge with ICU data, improving clinical decision modeling. It combines
graph representations with vital signs and clinical reports, enhancing
performance, especially when data is missing. Additionally, our model includes
an interpretability component to understand how knowledge graph nodes affect
predictions.

通过将医学领域的知识图谱与重症监护病房的数据进行整合，结合重要体征和临床报告，提高临床决策建模的性能，特别是在数据缺失时。同时包括一个可解释性组件，以了解知识图谱节点对预测的影响。