Skeleton Action Recognition (SAR) involves identifying human actions using
skeletal joint coordinates and their interconnections. While plain Transformers
have been attempted for this task, they still fall short compared to the
current leading methods, which are rooted in Graph Convolutional Networks
(GCNs) due to the absence of structural priors. Recently, a novel selective
state space model, Mamba, has surfaced as a compelling alternative to the
attention mechanism in Transformers, offering efficient modeling of long
sequences. In this work, to the utmost extent of our awareness, we present the
first SAR framework incorporating Mamba. Each fundamental block of our model
adopts a novel U-ShiftGCN architecture with Mamba as its core component. The
encoder segment of the U-ShiftGCN is devised to extract spatial features from
the skeletal data using downsampling vanilla Shift S-GCN blocks. These spatial
features then undergo intermediate temporal modeling facilitated by the Mamba
block before progressing to the encoder section, which comprises vanilla
upsampling Shift S-GCN blocks. Additionally, a Shift T-GCN (ShiftTCN) temporal
modeling unit is employed before the exit of each fundamental block to refine
temporal representations. This particular integration of downsampling spatial,
intermediate temporal, upsampling spatial, and ultimate temporal subunits
yields promising results for skeleton action recognition. We dub the resulting
model \textbf{Simba}, which attains state-of-the-art performance across three
well-known benchmark skeleton action recognition datasets: NTU RGB+D, NTU RGB+D
120, and Northwestern-UCLA. Interestingly, U-ShiftGCN (Simba without
Intermediate Mamba Block) by itself is capable of performing reasonably well
and surpasses our baseline.

本研究提出了一种将 Mamba 引入骨架动作识别的框架，命名为 Simba，通过采用 U-ShiftGCN 结构、Shift S-GCN 模块和 Shift T-GCN 模块，以及中间和最终时间建模单元的整合，实现了领先水平的骨架动作识别效果。