Medical image analysis is essential to clinical diagnosis and treatment,
which is increasingly supported by multi-modal large language models (MLLMs).
However, previous research has primarily focused on 2D medical images, leaving
3D images under-explored, despite their richer spatial information. This paper
aims to advance 3D medical image analysis with MLLMs. To this end, we present a
large-scale 3D multi-modal medical dataset, M3D-Data, comprising 120K
image-text pairs and 662K instruction-response pairs specifically tailored for
various 3D medical tasks, such as image-text retrieval, report generation,
visual question answering, positioning, and segmentation. Additionally, we
propose M3D-LaMed, a versatile multi-modal large language model for 3D medical
image analysis. Furthermore, we introduce a new 3D multi-modal medical
benchmark, M3D-Bench, which facilitates automatic evaluation across eight
tasks. Through comprehensive evaluation, our method proves to be a robust model
for 3D medical image analysis, outperforming existing solutions. All code,
data, and models are publicly available at: this https URL

该研究论文通过大规模的三维多模态医学数据集 M3D-Data 和多模态大型语言模型 M3D-LaMed，在各种三维医学任务上实现了先进的医学图像分析方法，并提出了用于自动评估的新的三维多模态医学基准 M3D-Bench。