Learning the spatial-temporal representation of motion information is crucial to human action recognition. Nevertheless, most of the existing features or descriptors cannot capture motion information effectively, especially for long-term motion. To address this problem, this paper proposes a long-term motion descriptor called sequential Deep Trajectory Descriptor (sDTD). Specifically, we project dense trajectories into two-dimensional planes, and subsequently a CNN-RNN network is employed to learn an effective representation for long-term motion. Unlike the popular two-stream ConvNets, the sDTD stream is introduced into a three-stream framework so as to identify actions from a video sequence. Consequently, this three-stream framework can simultaneously capture static spatial features, short-term motion and long-term motion in that video. Extensive experiments were conducted on three challenging datasets: KTH, HMDB51 and UCF101. Experimental results show that our method achieves state-of-the-art performance on the KTH and UCF101 datasets, and is comparable to the state-of-the-art methods on the HMDB51 dataset.

本文提出了一种名为连续深层轨迹描述符（sDTD）的长期运动描述符，它通过将稠密轨迹投影到二维平面上，再使用CNN-RNN网络来学习长期运动的有效表示，进而将sDTD流引入三流框架以从视频序列中识别动作。实验证明该方法在KTH和UCF101数据集上取得了最先进的性能，且在HMDB51数据集上与最先进的方法相媲美。

三流卷积神经网络中的连续深度轨迹描述符用于动作识别