Classical monocular Simultaneous Localization And Mapping (SLAM) and the
recently emerging convolutional neural networks (CNNs) for monocular depth
prediction represent two largely disjoint approaches towards building a 3D map
of the surrounding environment. In this paper, we demonstrate that the coupling
of these two by leveraging the strengths of each mitigates the other's
shortcomings. Specifically, we propose a joint narrow and wide baseline based
self-improving framework, where on the one hand the CNN-predicted depth is
leveraged to perform pseudo RGB-D feature-based SLAM, leading to better
accuracy and robustness than the monocular RGB SLAM baseline. On the other
hand, the bundle-adjusted 3D scene structures and camera poses from the more
principled geometric SLAM are injected back into the depth network through
novel wide baseline losses proposed for improving the depth prediction network,
which then continues to contribute towards better pose and 3D structure
estimation in the next iteration. We emphasize that our framework only requires
unlabeled monocular videos in both training and inference stages, and yet is
able to outperform state-of-the-art self-supervised monocular and stereo depth
prediction networks (e.g, Monodepth2) and feature-based monocular SLAM system
(i.e, ORB-SLAM). Extensive experiments on KITTI and TUM RGB-D datasets verify
the superiority of our self-improving geometry-CNN framework.

本文提出一种基于 SLAM 和 CNNs 的结合框架，将 CNN-predicted depth 用于伪 RGB-D feature-based SLAM，同时将几何 SLAM 的 3D 场景结构和相机姿势注入到深度网络中，以实现更好的姿态和 3D 结构估计。实验表明，该框架在 KITTI 和 TUM RGB-D 数据集上优于先前的方法。