The skeleton based gesture recognition is gaining more popularity due to its
wide possible applications. The key issues are how to extract discriminative
features and how to design the classification model. In this paper, we first
leverage a robust feature descriptor, path signature (PS), and propose three PS
features to explicitly represent the spatial and temporal motion
characteristics, i.e., spatial PS (S_PS), temporal PS (T_PS) and temporal
spatial PS (T_S_PS). Considering the significance of fine hand movements in the
gesture, we propose an "attention on hand" (AOH) principle to define joint
pairs for the S_PS and select single joint for the T_PS. In addition, the
dyadic method is employed to extract the T_PS and T_S_PS features that encode
global and local temporal dynamics in the motion. Secondly, without the
recurrent strategy, the classification model still faces challenges on temporal
variation among different sequences. We propose a new temporal transformer
module (TTM) that can match the sequence key frames by learning the temporal
shifting parameter for each input. This is a learning-based module that can be
included into standard neural network architecture. Finally, we design a
multi-stream fully connected layer based network to treat spatial and temporal
features separately and fused them together for the final result. We have
tested our method on three benchmark gesture datasets, i.e., ChaLearn 2016,
ChaLearn 2013 and MSRC-12. Experimental results demonstrate that we achieve the
state-of-the-art performance on skeleton-based gesture recognition with high
computational efficiency.

本研究提出了基于骨架的手势识别模型，采用重要的特征描述符 - 路径签名，并使用所提出的多种特征来明确表示空间和时间运动特征，并且通过注意手的原则来定义关节对，并使用 “dyadic method” 来提取 T_PS 和 T_S_PS 特征，最后采用多流全连接层和融合技术来进行分类，实验结果证明该方法可以实现骨架模型下手势识别的最佳表现。