Frame sampling is a fundamental problem in video action recognition due to
the essential redundancy in time and limited computation resources. The
existing sampling strategy often employs a fixed frame selection and lacks the
flexibility to deal with complex variations in videos. In this paper, we
present a simple, sparse, and explainable frame sampler, termed as
Motion-Guided Sampler (MGSampler). Our basic motivation is that motion is an
important and universal signal that can drive us to adaptively select frames
from videos. Accordingly, we propose two important properties in our MGSampler
design: motion sensitive and motion uniform. First, we present two different
motion representations to enable us to efficiently distinguish the
motion-salient frames from the background. Then, we devise a motion-uniform
sampling strategy based on the cumulative motion distribution to ensure the
sampled frames evenly cover all the important segments with high motion
salience. Our MGSampler yields a new principled and holistic sampling scheme,
that could be incorporated into any existing video architecture. Experiments on
five benchmarks demonstrate the effectiveness of our MGSampler over the
previous fixed sampling strategies, and its generalization power across
different backbones, video models, and datasets.

本文提出了一种基于运动敏感和运动一致性的框架抽取方法，称为 MGSampler，有效地解决了现有抽样策略中固定帧选择的不足，并在五大标准上进行了实验验证。