Recently, multi-modal vision-language foundation models have gained significant attention in the medical field. While these models offer great opportunities, they still face a number of challenges, such as the requirement for fine-grained knowledge understanding in computer-aided diagnosis and capability of utilizing very limited or no task-specific labeled data in real-world clinical applications. In this study, we present MaCo, a novel multi-modal medical foundation model that explores masked contrastive learning to achieve granular alignment and zero-shot learning for a variety of medical imaging tasks. MaCo incorporates a correlation weighting mechanism to adjust the correlation between masked image patches and their corresponding reports, thereby enhancing the representation learning capabilities. We evaluate MaCo on six well-known open-source X-ray datasets, and the experimental results show it outperforms seven state-of-the-art approaches for classification, segmentation, and zero-shot phase grounding, demonstrating its great potential to promote a wide range of medical image analysis tasks.

本研究介绍了MaCo，一种新颖的多模态医学基础模型，通过探索遮蔽对比学习实现医学影像任务的细粒度对齐和零样本学习，并结合相关权重机制来增强表示学习能力。我们在六个知名的开源X射线数据集上评估MaCo，实验结果表明其在分类、分割和零样本阶段定位方面优于七种最先进的方法，展示了其在促进各种医学图像分析任务方面的巨大潜力。

基于掩膜对比学习的细粒度对齐算法: 提升放射学报告基础模型的表示能力