In this work, we tackle the essential problem of scale inconsistency for
self-supervised joint depth-pose learning. Most existing methods assume that a
consistent scale of depth and pose can be learned across all input samples,
which makes the learning problem harder, resulting in degraded performance and
limited generalization in indoor environments and long-sequence visual odometry
application. To address this issue, we propose a novel system that explicitly
disentangles scale from the network estimation. Instead of relying on PoseNet
architecture, our method recovers relative pose by directly solving fundamental
matrix from dense optical flow correspondence and makes use of a two-view
triangulation module to recover an up-to-scale 3D structure. Then, we align the
scale of the depth prediction with the triangulated point cloud and use the
transformed depth map for depth error computation and dense reprojection check.
Our whole system can be jointly trained end-to-end. Extensive experiments show
that our system not only reaches state-of-the-art performance on KITTI depth
and flow estimation, but also significantly improves the generalization ability
of existing self-supervised depth-pose learning methods under a variety of
challenging scenarios, and achieves state-of-the-art results among
self-supervised learning-based methods on KITTI Odometry and NYUv2 dataset.
Furthermore, we present some interesting findings on the limitation of
PoseNet-based relative pose estimation methods in terms of generalization
ability. Code is available at this https URL

通过解决自监督联合深度姿态学习中的比例不一致问题，我们提出了一种新颖的系统，利用了三角化模块来恢复比例到三维结构，并使用变换后的深度图对深度误差进行计算和复制 Projection Check 。