Skeleton-based human action recognition has attracted a lot of research attention during the past few years. Recent works attempted to utilize recurrent neural networks to model the temporal dependencies between the 3D positional configurations of human body joints for better analysis of human activities in the skeletal data. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences in both of these domains simultaneously. Based on the pictorial structure of Kinect's skeletal data, an effective tree-structure based traversal framework is also proposed. In order to deal with the noise in the skeletal data, a new gating mechanism within LSTM module is introduced, with which the network can learn the reliability of the sequential data and accordingly adjust the effect of the input data on the updating procedure of the long-term context representation stored in the unit's memory cell. Moreover, we introduce a novel multi-modal feature fusion strategy within the LSTM unit in this paper. The comprehensive experimental results on seven challenging benchmark datasets for human action recognition demonstrate the effectiveness of the proposed method.

本论文提出了一种基于骨骼的人体动作识别方法，该方法采用循环神经网络建模体关节的3D位置配置之间的时间依赖关系，利用空间域和时间域共同分析骨架序列中与动作相关信息的潜在来源，并提出了一种有效的基于树状结构的遍历框架，以处理Kinect骨架数据中的噪声。该方法采用了新的LSTM模块中的门机制，通过学习连续数据的可靠性，并相应地调整长期上下文表示存储在单元的内存单元中输入数据的影响，从而处理骨架数据中的噪声，并提出了一种新颖的多模态特征融合策略。实验结果表明该方法在7个具有挑战性的基准数据集上具有很好的效果。

基于时空LSTM网络和可控门的骨架动作识别