Dominant approaches to action detection can only provide sub-optimal
solutions to the problem, as they rely on seeking frame-level detections, to
later compose them into "action tubes" in a post-processing step. With this
paper we radically depart from current practice, and take a first step towards
the design and implementation of a deep network architecture able to classify
and regress whole video subsets, so providing a truly optimal solution of the
action detection problem. In this work, in particular, we propose a novel deep
net framework able to regress and classify 3D region proposals spanning two
successive video frames, whose core is an evolution of classical region
proposal networks (RPNs). As such, our 3D-RPN net is able to effectively encode
the temporal aspect of actions by purely exploiting appearance, as opposed to
methods which heavily rely on expensive flow maps. The proposed model is
end-to-end trainable and can be jointly optimised for action localisation and
classification in a single step. At test time the network predicts
"micro-tubes" encompassing two successive frames, which are linked up into
complete action tubes via a new algorithm which exploits the temporal encoding
learned by the network and cuts computation time by 50%. Promising results on
the J-HMDB-21 and UCF-101 action detection datasets show that our model does
outperform the state-of-the-art when relying purely on appearance.

本论文提出了一种新的深度神经网络框架，称之为 3D-RPN 网络，能够在纯粹利用外观的情况下有效地编码动作的时间方面。这个模型是端到端可训练的，可以在单个步骤中联合优化动作定位和分类。在测试时，该网络预测了包含两个连续帧的微小管，而新算法则能够利用网络学习的时间编码来将它们组合成完整的动作管，计算时间减少了 50%。实验结果证明，该模型在仅利用外观的情况下优于同类方法，在 J-HMDB-21 和 UCF-101 动作检测数据集上表现良好。