While stochastic video prediction models enable future prediction under
uncertainty, they mostly fail to model the complex dynamics of real-world
scenes. For example, they cannot provide reliable predictions for scenes with a
moving camera and independently moving foreground objects in driving scenarios.
The existing methods fail to fully capture the dynamics of the structured world
by only focusing on changes in pixels. In this paper, we assume that there is
an underlying process creating observations in a video and propose to factorize
it into static and dynamic components. We model the static part based on the
scene structure and the ego-motion of the vehicle, and the dynamic part based
on the remaining motion of the dynamic objects. By learning separate
distributions of changes in foreground and background, we can decompose the
scene into static and dynamic parts and separately model the change in each.
Our experiments demonstrate that disentangling structure and motion helps
stochastic video prediction, leading to better future predictions in complex
driving scenarios on two real-world driving datasets, KITTI and Cityscapes.

本文提出了将视频观察结果分解为静态和动态部分，分别建模于场景结构和车辆主体运动、动态物体运动，来分离结构和运动的影响，从而更好地预测复杂驾驶场景中的未来运动。