In this paper, we propose MonoRec, a semi-supervised monocular dense reconstruction architecture that predicts depth maps from a single moving camera in dynamic environments. MonoRec is based on a MVS setting which encodes the information of multiple consecutive images in a cost volume. To deal with dynamic objects in the scene, we introduce a MaskModule that predicts moving object masks by leveraging the photometric inconsistencies encoded in the cost volumes. Unlike other MVS methods, MonoRec is able to predict accurate depths for both static and moving objects by leveraging the predicted masks. Furthermore, we present a novel multi-stage training scheme with a semi-supervised loss formulation that does not require LiDAR depth values. We carefully evaluate MonoRec on the KITTI dataset and show that it achieves state-of-the-art performance compared to both multi-view and single-view methods. With the model trained on KITTI, we further demonstrate that MonoRec is able to generalize well to both the Oxford RobotCar dataset and the more challenging TUM-Mono dataset recorded by a handheld camera. Training code and pre-trained model will be published soon.

本文提出了MonoRec，一种半监督单目稠密重建架构，可以在动态环境下从单个运动相机中预测深度地图。MonoRec基于多视图立体设置，通过成本体积编码多个连续图像的信息。为了处理场景中的动态物体，我们引入了MaskModule，通过利用成本体积中编码的光度不一致性来预测移动目标掩码。与其他多视图立体方法不同，MonoRec能够通过利用预测的掩码重建静态和移动物体。此外，我们提出了一种新的多阶段训练方案，其中使用半监督损失公式，不需要LiDAR深度值。我们在KITTI数据集上对MonoRec进行了认真评估，并表明它在与多视图和单视图方法相比具有最先进的性能。使用在KITTI上训练的模型，我们进一步证明了MonoRec能够很好地推广到牛津机器人车数据集和由手持相机记录的更具挑战性的TUM-Mono数据集。

MonoRec：单目移动相机在动态环境中的半监督稠密重建