Recent studies have shown that video-level representation learning is crucial
to the capture and understanding of the long-range temporal structure for video
action recognition. Most existing 3D convolutional neural network (CNN)-based
methods for video-level representation learning are clip-based and focus only
on short-term motion and appearances. These CNN-based methods lack the capacity
to incorporate and model the long-range spatiotemporal representation of the
underlying video and ignore the long-range video-level context during training.
In this study, we propose a factorized 4D CNN architecture with attention (F4D)
that is capable of learning more effective, finer-grained, long-term
spatiotemporal video representations. We demonstrate that the proposed F4D
architecture yields significant performance improvements over the conventional
2D, and 3D CNN architectures proposed in the literature. Experiment evaluation
on five action recognition benchmark datasets, i.e., Something-Something-v1,
SomethingSomething-v2, Kinetics-400, UCF101, and HMDB51 demonstrate the
effectiveness of the proposed F4D network architecture for video-level action
recognition.

通过提出具有注意力的分解 4D CNN 架构，文章研究了视频级别表示学习对于捕捉和理解视频动作识别中的长程时间结构的重要性，并在五个动作识别基准数据集上验证了所提出架构的有效性。