Conventional self-supervised monocular depth prediction methods are based on
a static environment assumption, which leads to accuracy degradation in dynamic
scenes due to the mismatch and occlusion problems introduced by object motions.
Existing dynamic-object-focused methods only partially solved the mismatch
problem at the training loss level. In this paper, we accordingly propose a
novel multi-frame monocular depth prediction method to solve these problems at
both the prediction and supervision loss levels. Our method, called
DynamicDepth, is a new framework trained via a self-supervised cycle consistent
learning scheme. A Dynamic Object Motion Disentanglement (DOMD) module is
proposed to disentangle object motions to solve the mismatch problem. Moreover,
novel occlusion-aware Cost Volume and Re-projection Loss are designed to
alleviate the occlusion effects of object motions. Extensive analyses and
experiments on the Cityscapes and KITTI datasets show that our method
significantly outperforms the state-of-the-art monocular depth prediction
methods, especially in the areas of dynamic objects. Code is available at
this https URL

这篇论文提出了一种自动学习的多帧单目深度预测方法 DynamicDepth，通过自补丁一致性学习方案训练一个新的框架，其中进行了动态物体运动分离来解决匹配问题，并设计了新型的遮挡感知代价卷积和复投影代价来减轻物体运动对遮挡的影响，实验证明该方法在动态物体方面显著优于现有的单目深度预测方法。

无监督多帧单目深度的物体运动和遮挡分离

Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth

Understanding, predicting, and generating object motions and transformations
is a core problem in artificial intelligence. Modeling sequences of evolving
images may provide better representations and models of motion and may
ultimately be used for forecasting, simulation, or video generation.
Diagrammatic Abstract Reasoning is an avenue in which diagrams evolve in
complex patterns and one needs to infer the underlying pattern sequence and
generate the next image in the sequence. For this, we develop a novel
Contextual Generative Adversarial Network based on Recurrent Neural Networks
(Context-RNN-GANs), where both the generator and the discriminator modules are
based on contextual history (modeled as RNNs) and the adversarial discriminator
guides the generator to produce realistic images for the particular time step
in the image sequence. We evaluate the Context-RNN-GAN model (and its variants)
on a novel dataset of Diagrammatic Abstract Reasoning, where it performs
competitively with 10th-grade human performance but there is still scope for
interesting improvements as compared to college-grade human performance. We
also evaluate our model on a standard video next-frame prediction task,
achieving improved performance over comparable state-of-the-art.

通过开发一种新的基于循环神经网络（RNN）的上下文生成敌对网络（Context-RNN-GANs）来通过建模逐步变化的图像序列，从而理解、预测和生成物体的运动和转换，它在图表抽象推理和标准视频下一帧预测任务中表现出与人类相似的水平。