We examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Our experiments reveal that while SAM demonstrates stunning segmentation performance on images from the general domain, for those out-of-distribution images, e.g., medical images, its zero-shot segmentation performance is still limited. Furthermore, SAM demonstrated varying zero-shot segmentation performance across different unseen medical domains. For example, it had a 0.8704 mean Dice score on segmenting under-bruch's membrane layer of retinal OCT, whereas the segmentation accuracy drops to 0.0688 when segmenting retinal pigment epithelium. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed, whereas a simple fine-tuning of it with small amount of data could lead to remarkable improvements of the segmentation quality. Our study indicates the versatility of generalist vision foundation models on solving specific tasks in medical imaging, and their great potential to achieve desired performance through fine-turning and eventually tackle the challenges of accessing large diverse medical datasets and the complexity of medical domains.

本研究使用Segment Anything Model在医疗图像上进行了定量和定性的零-shot分割实验，并发现虽然其在一般领域的图像中表现出色，但在医疗图像中存在一定的局限性，特别是在结构性目标上，而针对这些问题的数量有限的微调可以带来明显的改进。此研究表明了通用视觉基础模型在解决医疗成像中的特定任务方面的多功能性，并展现了这些模型通过微调最终可以应对获取大量不同的医学数据集和应对医学领域的复杂挑战的巨大潜力。

医学成像通用视觉基础模型：零样本医学分割中的“分割任意物体模型”案例研究