Existing methods to recognize actions in static images take the images at
their face value, learning the appearances---objects, scenes, and body
poses---that distinguish each action class. However, such models are deprived
of the rich dynamic structure and motions that also define human activity. We
propose an approach that hallucinates the unobserved future motion implied by a
single snapshot to help static-image action recognition. The key idea is to
learn a prior over short-term dynamics from thousands of unlabeled videos,
infer the anticipated optical flow on novel static images, and then train
discriminative models that exploit both streams of information. Our main
contributions are twofold. First, we devise an encoder-decoder convolutional
neural network and a novel optical flow encoding that can translate a static
image into an accurate flow map. Second, we show the power of hallucinated flow
for recognition, successfully transferring the learned motion into a standard
two-stream network for activity recognition. On seven datasets, we demonstrate
the power of the approach. It not only achieves state-of-the-art accuracy for
dense optical flow prediction, but also consistently enhances recognition of
actions and dynamic scenes.

通过一个 encoder-decoder CNN 和 optical flow 方法，我们的研究为静态图像动作识别提供了一个新方法，即学习一个短期动态先验，在新的静态图像上推断预期的光流，成功地将学习的动作转移至标准的两流网络中，并且在七个数据集上表现出了良好的性能。

静态图像的动作幻觉用于动作识别的 Im2Flow 算法

Im2Flow: Motion Hallucination from Static Images for Action Recognition

We propose a deep neural network for the prediction of future frames in
natural video sequences. To effectively handle complex evolution of pixels in
videos, we propose to decompose the motion and content, two key components
generating dynamics in videos. Our model is built upon the Encoder-Decoder
Convolutional Neural Network and Convolutional LSTM for pixel-level prediction,
which independently capture the spatial layout of an image and the
corresponding temporal dynamics. By independently modeling motion and content,
predicting the next frame reduces to converting the extracted content features
into the next frame content by the identified motion features, which simplifies
the task of prediction. Our model is end-to-end trainable over multiple time
steps, and naturally learns to decompose motion and content without separate
training. We evaluate the proposed network architecture on human activity
videos using KTH, Weizmann action, and UCF-101 datasets. We show
state-of-the-art performance in comparison to recent approaches. To the best of
our knowledge, this is the first end-to-end trainable network architecture with
motion and content separation to model the spatiotemporal dynamics for
pixel-level future prediction in natural videos.

提出了一种利用深度神经网络预测自然视频序列未来帧的方法，通过运动和内容分解进行像素级预测，该模型基于编码器 - 解码器卷积神经网络和卷积 LSTM，同时独立捕捉图像的空间布局和相应的时间动态，并在多个时间步长上进行端到端训练。