Activity detection is a fundamental problem in computer vision. Detecting activities of different temporal scales is particularly challenging. In this paper, we propose the contextual multi-scale region convolutional 3D network (CMS-RC3D) for activity detection. To deal with the inherent temporal scale variability of activity instances, the temporal feature pyramid is used to represent activities of different temporal scales. On each level of the temporal feature pyramid, an activity proposal detector and an activity classifier are learned to detect activities of specific temporal scales. Temporal contextual information is fused into activity classifiers for better recognition. More importantly, the entire model at all levels can be trained end-to-end. Our CMS-RC3D detector can deal with activities at all temporal scale ranges with only a single pass through the backbone network. We test our detector on two public activity detection benchmarks, THUMOS14 and ActivityNet. Extensive experiments show that the proposed CMS-RC3D detector outperforms state-of-the-art methods on THUMOS14 by a substantial margin and achieves comparable results on ActivityNet despite using a shallow feature extractor.

本文提出了一种用于活动检测的上下文多尺度区域卷积3D神经网络(CMS-RC3D)，通过使用时间特征金字塔来表示不同时间尺度的活动实例，并在每个时间特征金字塔的级别上都学习了一个针对特定时间尺度的活动建议检测器和活动分类器，以进行更好的识别。 实验结果表明，CMS-RC3D探测器在处理所有时间尺度范围内的活动时，仅需通过主干网络进行一次，而在THUMOS14数据集上比现有方法表现更好，并在ActivityNet数据集上实现了可比较的结果，尽管使用了浅层特征提取器。

用于活动检测的上下文多尺度区域卷积3D网络