We study the problem of synthesizing a number of likely future frames from a
single input image. In contrast to traditional methods that have tackled this
problem in a deterministic or non-parametric way, we propose to model future
frames in a probabilistic manner. Our probabilistic model makes it possible for
us to sample and synthesize many possible future frames from a single input
image. To synthesize realistic movement of objects, we propose a novel network
structure, namely a Cross Convolutional Network; this network encodes image and
motion information as feature maps and convolutional kernels, respectively. In
experiments, our model performs well on synthetic data, such as 2D shapes and
animated game sprites, and on real-world video frames. We present analyses of
the learned network representations, showing it is implicitly learning a
compact encoding of object appearance and motion. We also demonstrate a few of
its applications, including visual analogy-making and video extrapolation.

该研究提出采用概率建模的方法，基于图像和动态信息，使用交叉卷积网络结构来合成多个可能的未来帧，并能够实现物体运动的真实感合成和学习出物体的外观与运动的紧凑编码。同时，提出了可视类比和视频外推等应用方案。

视觉动力学：基于分层交叉卷积网络的随机未来生成

Visual Dynamics: Stochastic Future Generation via Layered Cross  Convolutional Networks

Boundary estimation in images and videos has been a very active topic of
research, and organizing visual information into boundaries and segments is
believed to be a corner stone of visual perception. While prior work has
focused on estimating boundaries for observed frames, our work aims at
predicting boundaries of future unobserved frames. This requires our model to
learn about the fate of boundaries and corresponding motion patterns --
including a notion of "intuitive physics". We experiment on natural video
sequences along with synthetic sequences with deterministic physics-based and
agent-based motions. While not being our primary goal, we also show that fusion
of RGB and boundary prediction leads to improved RGB predictions.

研究了边界估计和预测未来未观察到帧的边界能力，结合直观物理的概念，通过自然视频帧和合成序列的实验，发现 rgb 融合和边缘预测能够促进 rgb 预测的提升。