We propose GeoNet, a jointly unsupervised learning framework for monocular depth, optical flow and ego-motion estimation from videos. The three components are coupled by the nature of 3D scene geometry, jointly learned by our framework in an end-to-end manner. Specifically, geometric relationships are extracted over the predictions of individual modules and then combined as an image reconstruction loss, reasoning about static and dynamic scene parts separately. Furthermore, we propose an adaptive geometric consistency loss to increase robustness towards outliers and non-Lambertian regions, which resolves occlusions and texture ambiguities effectively. Experimentation on the KITTI driving dataset reveals that our scheme achieves state-of-the-art results in all of the three tasks, performing better than previously unsupervised methods and comparably with supervised ones.

本文提出了GeoNet，这是一个基于联合无监督学习的框架，可用于从视频中估计单眼深度、光流和Ego-motion。基于3D场景几何关系，由我们的框架以端到端的方式联合学习。实验结果表明，在KITTI驾驶数据集中，我们的方法在三个任务中均取得了最先进的成果，优于以前的无监督方法，并与受监督方法相当。

GeoNet: 密集深度、光流和相机位姿的无监督学习