The Segment Anything Model (SAM) is a powerful vision foundation model that
is revolutionizing the traditional paradigm of segmentation. Despite this, a
reliance on prompting each frame and large computational cost limit its usage
in robotically assisted surgery. Applications, such as augmented reality
guidance, require little user intervention along with efficient inference to be
usable clinically. In this study, we address these limitations by adopting
lightweight SAM variants to meet the speed requirement and employing
fine-tuning techniques to enhance their generalization in surgical scenes.
Recent advancements in Tracking Any Point (TAP) have shown promising results in
both accuracy and efficiency, particularly when points are occluded or leave
the field of view. Inspired by this progress, we present a novel framework that
combines an online point tracker with a lightweight SAM model that is
fine-tuned for surgical instrument segmentation. Sparse points within the
region of interest are tracked and used to prompt SAM throughout the video
sequence, providing temporal consistency. The quantitative results surpass the
state-of-the-art semi-supervised video object segmentation method on the
EndoVis 2015 dataset, with an over 25 FPS inference speed running on a single
GeForce RTX 4060 GPU.

在这项研究中，我们采用轻量级的 Segment Anything Model (SAM) 变体以满足速度要求，并采用微调技术来增强其在外科手术场景中的泛化能力。我们结合在线点追踪器和经过微调的轻量级 SAM 模型构建了一个新颖的框架，用于外科器械分割。在 EndoVis 2015 数据集上，定量结果超过了半监督视频对象分割方法的最新水平，并且可以在单个 GeForce RTX 4060 GPU 上以超过 25 FPS 的推理速度运行。