Motion plays a crucial role in understanding videos and most state-of-the-art
neural models for video classification incorporate motion information typically
using optical flows extracted by a separate off-the-shelf method. As the
frame-by-frame optical flows require heavy computation, incorporating motion
information has remained a major computational bottleneck for video
understanding. In this work, we replace external and heavy computation of
optical flows with internal and light-weight learning of motion features. We
propose a trainable neural module, dubbed MotionSqueeze, for effective motion
feature extraction. Inserted in the middle of any neural network, it learns to
establish correspondences across frames and convert them into motion features,
which are readily fed to the next downstream layer for better prediction. We
demonstrate that the proposed method provides a significant gain on four
standard benchmarks for action recognition with only a small amount of
additional cost, outperforming the state of the art on
Something-Something-V1&V2 datasets.

本文提出了一种名为 MotionSqueeze 的可训练神经网络模块，用于有效的运动特征提取，将外部和重量级计算的光流替换为内部和轻量级的学习运动特征方法，在四个标准基准测试中，该方法仅需很少的额外成本，即可获得明显的提高，超越了 Something-Something-V1&V2 数据集上的现有技术。