Capturing the dependencies between joints is critical in skeleton-based
action recognition task. Transformer shows great potential to model the
correlation of important joints. However, the existing Transformer-based
methods cannot capture the correlation of different joints between frames,
which the correlation is very useful since different body parts (such as the
arms and legs in "long jump") between adjacent frames move together. Focus on
this problem, A novel spatio-temporal tuples Transformer (STTFormer) method is
proposed. The skeleton sequence is divided into several parts, and several
consecutive frames contained in each part are encoded. And then a
spatio-temporal tuples self-attention module is proposed to capture the
relationship of different joints in consecutive frames. In addition, a feature
aggregation module is introduced between non-adjacent frames to enhance the
ability to distinguish similar actions. Compared with the state-of-the-art
methods, our method achieves better performance on two large-scale datasets.

提出了一种新的时空元组 Transformer（STTFormer）方法，该方法着重解决了现有的基于 Transformer 的方法不能捕获帧之间不同关节之间的相关性的问题，在两个大型数据集上的表现优于现有技术方法。