While learning based depth estimation from images/videos has achieved
substantial progress, there still exist intrinsic limitations. Supervised
methods are limited by a small amount of ground truth or labeled data and
unsupervised methods for monocular videos are mostly based on the static scene
assumption, not performing well on real world scenarios with the presence of
dynamic objects. In this paper, we propose a new learning based method
consisting of DepthNet, PoseNet and Region Deformer Networks (RDN) to estimate
depth from unconstrained monocular videos without ground truth supervision. The
core contribution lies in RDN for proper handling of rigid and non-rigid
motions of various objects such as rigidly moving cars and deformable humans.
In particular, a deformation based motion representation is proposed to model
individual object motion on 2D images. This representation enables our method
to be applicable to diverse unconstrained monocular videos. Our method can not
only achieve the state-of-the-art results on standard benchmarks KITTI and
Cityscapes, but also show promising results on a crowded pedestrian tracking
dataset, which demonstrates the effectiveness of the deformation based motion
representation. Code and trained models are available at
this https URL

本文提出了一种新的基于学习的深度估计方法，由 DepthNet、PoseNet 和 Region Deformer Networks（RDN）组成，用于无约束单眼视频的深度估计，其核心贡献在于 RDN 对刚性和非刚性物体的适当处理，特别是提出了基于变形的运动表示来在 2D 图像上对个体运动建模，该表示使我们的方法适用于多样化和无约束的单眼视频，并分别在 KITTI 和 Cityscapes 基准测试以及人行追踪数据集上展示了非常好的结果。