Scene flow estimation has been receiving increasing attention for 3D environment perception. Monocular scene flow estimation -- obtaining 3D structure and 3D motion from two temporally consecutive images -- is a highly ill-posed problem, and practical solutions are lacking to date. We propose a novel monocular scene flow method that yields competitive accuracy and real-time performance. By taking an inverse problem view, we design a single convolutional neural network (CNN) that successfully estimates depth and 3D motion simultaneously from a classical optical flow cost volume. We adopt self-supervised learning with 3D loss functions and occlusion reasoning to leverage unlabeled data. We validate our design choices, including the proxy loss and augmentation setup. Our model achieves state-of-the-art accuracy among unsupervised/self-supervised learning approaches to monocular scene flow, and yields competitive results for the optical flow and monocular depth estimation sub-tasks. Semi-supervised fine-tuning further improves the accuracy and yields promising results in real-time.

本文提出的单目场景流（Scene Flow）估计方法基于一个卷积神经网络（CNN），通过充分考虑光流代价体积，成功估算出深度和三维运动。我们采用了自监督学习方法，利用三维损失函数和遮挡推理来提高估计精度。实验结果表明，该方法在单目场景流估计领域取得了最佳性能，同时在光流和单目深度估计子任务上也获得了很好的结果。

自监督单目场景流估计