Existing methods in video action recognition mostly do not distinguish human
body from the environment and easily overfit the scenes and objects. In this
work, we present a conceptually simple, general and high-performance framework
for action recognition in trimmed videos, aiming at person-centric modeling.
The method, called Action Machine, takes as inputs the videos cropped by person
bounding boxes. It extends the Inflated 3D ConvNet (I3D) by adding a branch for
human pose estimation and a 2D CNN for pose-based action recognition, being
fast to train and test. Action Machine can benefit from the multi-task training
of action recognition and pose estimation, the fusion of predictions from RGB
images and poses. On NTU RGB-D, Action Machine achieves the state-of-the-art
performance with top-1 accuracies of 97.2% and 94.3% on cross-view and
cross-subject respectively. Action Machine also achieves competitive
performance on another three smaller action recognition datasets: Northwestern
UCLA Multiview Action3D, MSR Daily Activity3D and UTD-MHAD. Code will be made
available.

本文提出了一种名为 Action Machine 的基于人物框架的行为识别方法，通过将 Inflated 3D ConvNet (I3D) 与人体姿态估计和基于姿态的行为识别相结合，实现了在裁剪视频中的高性能行为识别，并在多个数据集上取得了 State-of-the-Art 的表现。

动作机器：重新思考裁剪视频中的动作识别

Action Machine: Rethinking Action Recognition in Trimmed Videos

The paucity of videos in current action classification datasets (UCF-101 and
HMDB-51) has made it difficult to identify good video architectures, as most
methods obtain similar performance on existing small-scale benchmarks. This
paper re-evaluates state-of-the-art architectures in light of the new Kinetics
Human Action Video dataset. Kinetics has two orders of magnitude more data,
with 400 human action classes and over 400 clips per class, and is collected
from realistic, challenging YouTube videos. We provide an analysis on how
current architectures fare on the task of action classification on this dataset
and how much performance improves on the smaller benchmark datasets after
pre-training on Kinetics.
We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on
2D ConvNet inflation: filters and pooling kernels of very deep image
classification ConvNets are expanded into 3D, making it possible to learn
seamless spatio-temporal feature extractors from video while leveraging
successful ImageNet architecture designs and even their parameters. We show
that, after pre-training on Kinetics, I3D models considerably improve upon the
state-of-the-art in action classification, reaching 80.9% on HMDB-51 and 98.0%
on UCF-101.

本研究基于 Kinetics 数据集重新评估最先进的体系结构，并引入一种新的双流膨胀 3D ConvNet（I3D），该 ConvNet 可以在视频中学习无缝的时空特征提取器，利用成功的 ImageNet 架构设计及其参数，经过在 Kinetics 上的预训练后，I3D 模型在动作分类方面表现明显提高。