Self-supervised monocular depth estimation presents a powerful method to
obtain 3D scene information from single camera images, which is trainable on
arbitrary image sequences without requiring depth labels, e.g., from a LiDAR
sensor. In this work we present a new self-supervised semantically-guided depth
estimation (SGDepth) method to deal with moving dynamic-class (DC) objects,
such as moving cars and pedestrians, which violate the static-world assumptions
typically made during training of such models. Specifically, we propose (i)
mutually beneficial cross-domain training of (supervised) semantic segmentation
and self-supervised depth estimation with task-specific network heads, (ii) a
semantic masking scheme providing guidance to prevent moving DC objects from
contaminating the photometric loss, and (iii) a detection method for frames
with non-moving DC objects, from which the depth of DC objects can be learned.
We demonstrate the performance of our method on several benchmarks, in
particular on the Eigen split, where we exceed all baselines without test-time
refinement.

本文提出了一种自主引导的语义深度估计方法（SGDepth），通过交叉域训练避免移动车辆和行人等动态类别对象对深度标签的干扰，利用异构网络头进行语义分割和深度估计，采用语义遮罩机制预防动态物体的污染，并引入一种检测算法学习移动类别对象的深度。通过在多个数据集上评估验证了该方法的性能。