Data in the healthcare domain arise from a variety of sources and modalities, such as x-ray images, continuous measurements, and clinical notes. Medical practitioners integrate these diverse data types daily to make informed and accurate decisions. With recent advancements in language models capable of handling multimodal data, it is a logical progression to apply these models to the healthcare sector. In this work, we introduce a framework that connects small language models to multiple data sources, aiming to predict the risk of various diseases simultaneously. Our experiments encompass 12 different tasks within a multitask learning setup. Although our approach does not surpass state-of-the-art methods specialized for single tasks, it demonstrates competitive performance and underscores the potential of small language models for multimodal reasoning in healthcare.

本研究解决了医疗领域中多种数据来源和模式整合不足的问题，提出了一种将小型语言模型与多种数据源结合的框架，以实现多种疾病风险的同步预测。虽然该方法的表现未超越专注单一任务的先进技术，但表明了小型语言模型在医疗多模态推理中的巨大潜力。

基于小型语言模型的全面疾病风险预测