Monocular depth estimation in endoscopy videos can enable assistive and
robotic surgery to obtain better coverage of the organ and detection of various
health issues. Despite promising progress on mainstream, natural image depth
estimation, techniques perform poorly on endoscopy images due to a lack of
strong geometric features and challenging illumination effects. In this paper,
we utilize the photometric cues, i.e., the light emitted from an endoscope and
reflected by the surface, to improve monocular depth estimation. We first
create two novel loss functions with supervised and self-supervised variants
that utilize a per-pixel shading representation. We then propose a novel depth
refinement network (PPSNet) that leverages the same per-pixel shading
representation. Finally, we introduce teacher-student transfer learning to
produce better depth maps from both synthetic data with supervision and
clinical data with self-supervision. We achieve state-of-the-art results on the
C3VD dataset while estimating high-quality depth maps from clinical data. Our
code, pre-trained models, and supplementary materials can be found on our
project page: this https URL

本论文介绍了在内窥镜图像中利用光度线索提高单目深度估计的方法，并通过使用自监督和监督训练的方法，结合教师 - 学生迁移学习，实现了对内窥镜图像的高质量深度估计。

利用近场照明从内窥镜视频进行单目深度估计

Leveraging Near-Field Lighting for Monocular Depth Estimation from  Endoscopy Videos

Dense 3D reconstruction and ego-motion estimation are key challenges in
autonomous driving and robotics. Compared to the complex, multi-modal systems
deployed today, multi-camera systems provide a simpler, low-cost alternative.
However, camera-based 3D reconstruction of complex dynamic scenes has proven
extremely difficult, as existing solutions often produce incomplete or
incoherent results. We propose R3D3, a multi-camera system for dense 3D
reconstruction and ego-motion estimation. Our approach iterates between
geometric estimation that exploits spatial-temporal information from multiple
cameras, and monocular depth refinement. We integrate multi-camera feature
correlation and dense bundle adjustment operators that yield robust geometric
depth and pose estimates. To improve reconstruction where geometric depth is
unreliable, e.g. for moving objects or low-textured regions, we introduce
learnable scene priors via a depth refinement network. We show that this design
enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor
environments. Consequently, we achieve state-of-the-art dense depth prediction
on the DDAD and NuScenes benchmarks.

提出了一种多摄像机系统来实现密集的三维重建和自我运动估计，通过多摄像机间的几何估计和单目深度优化来获得鲁棒的几何深度和姿态估计，同时通过深度优化网络引入可学习的场景先验，从而在具有挑战性的动态室外环境中实现了稠密、一致的三维重建。