Segment anything model (SAM), a foundation model with superior versatility
and generalization across diverse segmentation tasks, has attracted widespread
attention in medical imaging. However, it has been proved that SAM would
encounter severe performance degradation due to the lack of medical knowledge
in training and local feature encoding. Though several SAM-based models have
been proposed for tuning SAM in medical imaging, they still suffer from
insufficient feature extraction and highly rely on high-quality prompts. In
this paper, we construct a large CT dataset consisting of 1.1M CT images and 5M
masks from public datasets and propose a powerful foundation model SAMCT
allowing labor-free prompts. Specifically, based on SAM, SAMCT is further
equipped with a U-shaped CNN image encoder, a cross-branch interaction module,
and a task-indicator prompt encoder. The U-shaped CNN image encoder works in
parallel with the ViT image encoder in SAM to supplement local features.
Cross-branch interaction enhances the feature expression capability of the CNN
image encoder and the ViT image encoder by exchanging global perception and
local features from one to the other. The task-indicator prompt encoder is a
plug-and-play component to effortlessly encode task-related indicators into
prompt embeddings. In this way, SAMCT can work in an automatic manner in
addition to the semi-automatic interactive strategy in SAM. Extensive
experiments demonstrate the superiority of SAMCT against the state-of-the-art
task-specific and SAM-based medical foundation models on various tasks. The
code, data, and models are released at this https URL

该研究提出了一个强大的基础模型 SAMCT，通过 U-shaped CNN 图像编码器、交叉分支交互模块和任务指示器提示编码器的使用，实现了对医学成像中各种任务的自动化操作，并在相关任务上展示了其相对于现有模型的优越性。