The Segment Anything Model (SAM) is a powerful foundation model that has
revolutionised image segmentation. To apply SAM to surgical instrument
segmentation, a common approach is to locate precise points or boxes of
instruments and then use them as prompts for SAM in a zero-shot manner.
However, we observe two problems with this naive pipeline: (1) the domain gap
between natural objects and surgical instruments leads to poor generalisation
of SAM; and (2) SAM relies on precise point or box locations for accurate
segmentation, requiring either extensive manual guidance or a well-performing
specialist detector for prompt preparation, which leads to a complex
multi-stage pipeline. To address these problems, we introduce SurgicalSAM, a
novel end-to-end efficient-tuning approach for SAM to effectively integrate
surgical-specific information with SAM's pre-trained knowledge for improved
generalisation. Specifically, we propose a lightweight prototype-based class
prompt encoder for tuning, which directly generates prompt embeddings from
class prototypes and eliminates the use of explicit prompts for improved
robustness and a simpler pipeline. In addition, to address the low inter-class
variance among surgical instrument categories, we propose contrastive prototype
learning, further enhancing the discrimination of the class prototypes for more
accurate class prompting. The results of extensive experiments on both
EndoVis2018 and EndoVis2017 datasets demonstrate that SurgicalSAM achieves
state-of-the-art performance while only requiring a small number of tunable
parameters. The source code will be released at
this https URL

SurgicalSAM 是一种针对显著改善 SAM 泛化性能的新型端到端高效调优方法，引入了轻量级基于原型的类提示编码器，并提出了对比原型学习以进一步增强类原型的区分力，通过在 EndoVis2018 和 EndoVis2017 数据集上进行广泛实验，证明其达到了最先进的性能，仅需要少量可调参数。