Instance segmentation of surgical instruments is a long-standing research
problem, crucial for the development of many applications for computer-assisted
surgery. This problem is commonly tackled via fully-supervised training of deep
learning models, requiring expensive pixel-level annotations to train. In this
work, we develop a framework for instance segmentation not relying on spatial
annotations for training. Instead, our solution only requires binary tool
masks, obtainable using recent unsupervised approaches, and binary tool
presence labels, freely obtainable in robot-assisted surgery. Based on the
binary mask information, our solution learns to extract individual tool
instances from single frames, and to encode each instance into a compact vector
representation, capturing its semantic features. Such representations guide the
automatic selection of a tiny number of instances (8 only in our experiments),
displayed to a human operator for tool-type labelling. The gathered information
is finally used to match each training instance with a binary tool presence
label, providing an effective supervision signal to train a tool instance
classifier. We validate our framework on the EndoVis 2017 and 2018 segmentation
datasets. We provide results using binary masks obtained either by manual
annotation or as predictions of an unsupervised binary segmentation model. The
latter solution yields an instance segmentation approach completely free from
spatial annotations, outperforming several state-of-the-art fully-supervised
segmentation approaches.

通过仅使用二进制工具遮罩以及机器人辅助手术中的二进制工具存在标签，我们开发了一个无需空间注释的实例分割框架，该框架能从单帧中提取个体工具实例并进行语义特征编码，使得只需以人类操作员进行工具类别标记的方式便能有效训练工具实例分类器。通过验证我们的框架在 EndoVis 2017 和 2018 分割数据集上，我们发现与几种最先进的全监督分割方法相比，通过手动注释或基于无监督二进制分割模型的预测获得的二进制遮罩的实例分割方法无需空间注释且性能更优。

SAF-IS：一种用于手术工具实例分割的无需空间注释的框架

SAF-IS: a Spatial Annotation Free Framework for Instance Segmentation of  Surgical Tools

State-of-the-art research of traditional computer vision is increasingly
leveraged in the surgical domain. A particular focus in computer-assisted
surgery is to replace marker-based tracking systems for instrument localization
with pure image-based 6DoF pose estimation. However, the state of the art has
not yet met the accuracy required for surgical navigation. In this context, we
propose a high-fidelity marker-less optical tracking system for surgical
instrument localization. We developed a multi-view camera setup consisting of
static and mobile cameras and collected a large-scale RGB-D video dataset with
dedicated synchronization and data fusions methods. Different state-of-the-art
pose estimation methods were integrated into a deep learning pipeline and
evaluated on multiple camera configurations. Furthermore, the performance
impacts of different input modalities and camera positions, as well as training
on purely synthetic data, were compared. The best model achieved an average
position and orientation error of 1.3 mm and 1.0{\deg} for a surgical drill as
well as 3.8 mm and 5.2{\deg} for a screwdriver. These results significantly
outperform related methods in the literature and are close to clinical-grade
accuracy, demonstrating that marker-less tracking of surgical instruments is
becoming a feasible alternative to existing marker-based systems.

提出了一种高保真度的非标记光学跟踪系统，用于手术器械的本地化，采用多视角相机布置，利用深度学习方法和数据融合技术，对不同输入模型和相机位置进行评估和比较，并且取得了比文献中相关方法更好的平均误差结果，从而展示了手术器械无标记跟踪已经成为了现实。

下一代手术导航：手术器械多视图无标记 6 自由度姿态估计

Next-generation Surgical Navigation: Multi-view Marker-less 6DoF Pose  Estimation of Surgical Instruments

Intraoperative segmentation and tracking of minimally invasive instruments is
a prerequisite for computer- and robotic-assisted surgery. Since additional
hardware like tracking systems or the robot encoders are cumbersome and lack
accuracy, surgical vision is evolving as promising techniques to segment and
track the instruments using only the endoscopic images. However, what is
missing so far are common image data sets for consistent evaluation and
benchmarking of algorithms against each other. The paper presents a comparative
validation study of different vision-based methods for instrument segmentation
and tracking in the context of robotic as well as conventional laparoscopic
surgery. The contribution of the paper is twofold: we introduce a comprehensive
validation data set that was provided to the study participants and present the
results of the comparative validation study. Based on the results of the
validation study, we arrive at the conclusion that modern deep learning
approaches outperform other methods in instrument segmentation tasks, but the
results are still not perfect. Furthermore, we show that merging results from
different methods actually significantly increases accuracy in comparison to
the best stand-alone method. On the other hand, the results of the instrument
tracking task show that this is still an open challenge, especially during
challenging scenarios in conventional laparoscopic surgery.

本文介绍了一项比较研究，探讨了在机器人以及传统腹腔镜手术中，基于视觉的不同方法用于器械分割和追踪的效果。研究表明，现代的深度学习方法在器械分割任务中表现优异，但结果还不完美。此外，合并不同方法的结果实际上显著提高了准确性。与此同时，器械追踪任务的结果表明，在传统腹腔镜手术中这仍然是一个待解决的挑战。