Unsupervised methods have showed promising results on monocular depth estimation. However, the training data must be captured in scenes without moving objects. To push the envelope of accuracy, recent methods tend to increase their model parameters. In this paper, an unsupervised learning framework is proposed to jointly predict monocular depth and complete 3D motion including the motions of moving objects and camera. (1) Recurrent modulation units are used to adaptively and iteratively fuse encoder and decoder features. This not only improves the single-image depth inference but also does not overspend model parameters. (2) Instead of using a single set of filters for upsampling, multiple sets of filters are devised for the residual upsampling. This facilitates the learning of edge-preserving filters and leads to the improved performance. (3) A warping-based network is used to estimate a motion field of moving objects without using semantic priors. This breaks down the requirement of scene rigidity and allows to use general videos for the unsupervised learning. The motion field is further regularized by an outlier-aware training loss. Despite the depth model just uses a single image in test time and 2.97M parameters, it achieves state-of-the-art results on the KITTI and Cityscapes benchmarks.

该论文提出了一种无监督学习框架，通过使用逐步的递归调整和融合编码器和解码器特征来提高单张图像的深度推断，并使用多组滤波器用于残差上采样来学习保持边缘的滤波，使用基于自运动恢复的网络来估计移动对象的移动场，并进一步通过采用鲁棒训练损失对模型中参数的使用进行优化，该模型结构只使用单张图像在测试时间中，模型参数仅有2.97百万个，但却在KITTI和Cityscapes基准测试中实现了最先进的结果。

RM-Depth：动态场景下基于循环结构的单目深度非监督学习