Skeleton-based human action recognition has achieved a great interest in
recent years, as skeleton data has been demonstrated to be robust to
illumination changes, body scales, dynamic camera views, and complex
background. Nevertheless, an effective encoding of the latent information
underlying the 3D skeleton is still an open problem. In this work, we propose a
novel Spatial-Temporal Transformer network (ST-TR) which models dependencies
between joints using the Transformer self-attention operator. In our ST-TR
model, a Spatial Self-Attention module (SSA) is used to understand intra-frame
interactions between different body parts, and a Temporal Self-Attention module
(TSA) to model inter-frame correlations. The two are combined in a two-stream
network which outperforms state-of-the-art models using the same input data on
both NTU-RGB+D 60 and NTU-RGB+D 120.

本文提出了一种新的空时变换器网络（ST-TR），该网络利用 Transformer 自我注意机制对 3D 骨架中的关节之间的依赖关系进行建模。空间自我注意模块（SSA）用于理解不同身体部位之间的帧内相互作用，而时间自我注意模块（TSA）用于建模帧间的相关性。两个模块在两个流网络中被结合使用，对 NTU-RGB + D 60 和 NTU-RGB + D 120 上的相同输入数据优于同类基准模型。