Current state-of-the-art approaches for spatio-temporal action localization
rely on detections at the frame level that are then linked or tracked across
time. In this paper, we leverage the temporal continuity of videos instead of
operating at the frame level. We propose the ACtion Tubelet detector
(ACT-detector) that takes as input a sequence of frames and outputs tubelets,
i.e., sequences of bounding boxes with associated scores. The same way
state-of-the-art object detectors rely on anchor boxes, our ACT-detector is
based on anchor cuboids. We build upon the SSD framework. Convolutional
features are extracted for each frame, while scores and regressions are based
on the temporal stacking of these features, thus exploiting information from a
sequence. Our experimental results show that leveraging sequences of frames
significantly improves detection performance over using individual frames. The
gain of our tubelet detector can be explained by both more accurate scores and
more precise localization. Our ACT-detector outperforms the state-of-the-art
methods for frame-mAP and video-mAP on the J-HMDB and UCF-101 datasets, in
particular at high overlap thresholds.

本文通过提出 ACT-detector，结合对象检测中的 anchor boxes 思想，利用视频的时间连续性进行动作检测，结果表明在 J-HMDB 和 UCF-101 数据集上，我们的 ACT-detector 检测方法在帧 - 平均精度 (frame-mAP) 和视频 - 平均精度 (video-mAP) 上都优于基于帧的最新方法，特别是具有高重叠阈值时，精度和定位都得到了显著的提升。