Visual signals in a video can be divided into content and motion. While
content specifies which objects are in the video, motion describes their
dynamics. Based on this prior, we propose the Motion and Content decomposed
Generative Adversarial Network (MoCoGAN) framework for video generation. The
proposed framework generates a video by mapping a sequence of random vectors to
a sequence of video frames. Each random vector consists of a content part and a
motion part. While the content part is kept fixed, the motion part is realized
as a stochastic process. To learn motion and content decomposition in an
unsupervised manner, we introduce a novel adversarial learning scheme utilizing
both image and video discriminators. Extensive experimental results on several
challenging datasets with qualitative and quantitative comparison to the
state-of-the-art approaches, verify effectiveness of the proposed framework. In
addition, we show that MoCoGAN allows one to generate videos with same content
but different motion as well as videos with different content and same motion.

本文提出了一种 MoCoGAN 框架，利用运动和内容分解，通过映射一系列随机向量到一系列视频帧来生成视频，采用新型的对抗学习方案，使用图像和视频鉴别器，进行无监督学习，实现运动和内容分解。实验证明了 MoCoGAN 框架具有出色的效果，它不仅可以生成内容相同但运动不同的视频，还可以生成内容不同但运动相同的视频。

MoCoGAN: 视频生成的运动与内容分解

MoCoGAN: Decomposing Motion and Content for Video Generation

We propose a deep neural network for the prediction of future frames in
natural video sequences. To effectively handle complex evolution of pixels in
videos, we propose to decompose the motion and content, two key components
generating dynamics in videos. Our model is built upon the Encoder-Decoder
Convolutional Neural Network and Convolutional LSTM for pixel-level prediction,
which independently capture the spatial layout of an image and the
corresponding temporal dynamics. By independently modeling motion and content,
predicting the next frame reduces to converting the extracted content features
into the next frame content by the identified motion features, which simplifies
the task of prediction. Our model is end-to-end trainable over multiple time
steps, and naturally learns to decompose motion and content without separate
training. We evaluate the proposed network architecture on human activity
videos using KTH, Weizmann action, and UCF-101 datasets. We show
state-of-the-art performance in comparison to recent approaches. To the best of
our knowledge, this is the first end-to-end trainable network architecture with
motion and content separation to model the spatiotemporal dynamics for
pixel-level future prediction in natural videos.

提出了一种利用深度神经网络预测自然视频序列未来帧的方法，通过运动和内容分解进行像素级预测，该模型基于编码器 - 解码器卷积神经网络和卷积 LSTM，同时独立捕捉图像的空间布局和相应的时间动态，并在多个时间步长上进行端到端训练。