Depth estimation plays a crucial role in various tasks within endoscopic
surgery, including navigation, surface reconstruction, and augmented reality
visualization. Despite the significant achievements of foundation models in
vision tasks, including depth estimation, their direct application to the
medical domain often results in suboptimal performance. This highlights the
need for efficient adaptation methods to adapt these models to endoscopic depth
estimation. We propose Endoscopic Depth Any Camera (EndoDAC) which is an
efficient self-supervised depth estimation framework that adapts foundation
models to endoscopic scenes. Specifically, we develop the Dynamic Vector-Based
Low-Rank Adaptation (DV-LoRA) and employ Convolutional Neck blocks to tailor
the foundational model to the surgical domain, utilizing remarkably few
trainable parameters. Given that camera information is not always accessible,
we also introduce a self-supervised adaptation strategy that estimates camera
intrinsics using the pose encoder. Our framework is capable of being trained
solely on monocular surgical videos from any camera, ensuring minimal training
costs. Experiments demonstrate that our approach obtains superior performance
even with fewer training epochs and unaware of the ground truth camera
intrinsics. Code is available at this https URL

我们提出了一种高效的自监督深度估计框架 EndoDAC，它通过采用动态向量化低秩适应方法（DV-LoRA）和卷积颈块，利用极少的可训练参数将基础模型适应于外科手术领域，从而在更少的训练时期和未了解相机内参数的情况下获得卓越的性能。

EndoDAC: 适用于任何内视镜摄像机的高效自我监督深度估计的适应性基础模型

EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth  Estimation from Any Endoscopic Camera

Endoscopic surgery is currently an important treatment method in the field of
spinal surgery and avoiding damage to the spinal nerves through video guidance
is a key challenge. This paper presents the first real-time segmentation method
for spinal nerves in endoscopic surgery, which provides crucial navigational
information for surgeons. A finely annotated segmentation dataset of
approximately 10,000 consec-utive frames recorded during surgery is constructed
for the first time for this field, addressing the problem of semantic
segmentation. Based on this dataset, we propose FUnet (Frame-Unet), which
achieves state-of-the-art performance by utilizing inter-frame information and
self-attention mechanisms. We also conduct extended exper-iments on a similar
polyp endoscopy video dataset and show that the model has good generalization
ability with advantageous performance. The dataset and code of this work are
presented at: this https URL .

本文介绍了一种用于脊柱手术的首个实时分割方法，通过视频引导避免对脊神经的损伤，为外科医生提供重要的导航信息。此方法基于一份约 10,000 帧的细致注释分割数据集，针对语义分割问题，提出了 FUnet (Frame-Unet) 模型，通过利用帧间信息和自我注意机制，实现了业界最先进的性能。我们还对相似的息肉内窥镜视频数据集进行了拓展实验，结果表明该模型具有良好的泛化能力和优越的性能。本文提供了数据集和代码，详见此 https URL。

腔镜手术场景下的脊神经分割方法和数据集构建

Spinal nerve segmentation method and dataset construction in endoscopic  surgical scenarios

Traditional control and task automation have been successfully demonstrated
in a variety of structured, controlled environments through the use of highly
specialized modeled robotic systems in conjunction with multiple sensors.
However, the application of autonomy in endoscopic surgery is very challenging,
particularly in soft tissue work, due to the lack of high-quality images and
the unpredictable, constantly deforming environment. In this work, we propose a
novel surgical perception framework, SuPer, for surgical robotic control. This
framework continuously collects 3D geometric information that allows for
mapping a deformable surgical field while tracking rigid instruments within the
field. To achieve this, a model-based tracker is employed to localize the
surgical tool with a kinematic prior in conjunction with a model-free tracker
to reconstruct the deformable environment and provide an estimated point cloud
as a mapping of the environment. The proposed framework was implemented on the
da Vinci Surgical System in real-time with an end-effector controller where the
target configurations are set and regulated through the framework. Our proposed
framework successfully completed soft tissue manipulation tasks with high
accuracy. The demonstration of this novel framework is promising for the future
of surgical autonomy. In addition, we provide our dataset for further surgical
research.

本研究提出了一种新颖的外科手术知觉框架 SuPer，该框架使用基于模型的追踪器与无模型追踪器相结合，实时地收集 3D 几何信息并构建可变形手术视野，成功地实现了对软组织的精确操控任务。同时，我们将提供本研究的数据集用于未来的外科手术研究。