Graph Convolutional Networks (GCNs) have been widely used to model the
high-order dynamic dependencies for skeleton-based action recognition. Most
existing approaches do not explicitly embed the high-order spatio-temporal
importance to joints' spatial connection topology and intensity, and they do
not have direct objectives on their attention module to jointly learn when and
where to focus on in the action sequence. To address these problems, we propose
the To-a-T Spatio-Temporal Focus (STF), a skeleton-based action recognition
framework that utilizes the spatio-temporal gradient to focus on relevant
spatio-temporal features. We first propose the STF modules with learnable
gradient-enforced and instance-dependent adjacency matrices to model the
high-order spatio-temporal dynamics. Second, we propose three loss terms
defined on the gradient-based spatio-temporal focus to explicitly guide the
classifier when and where to look at, distinguish confusing classes, and
optimize the stacked STF modules. STF outperforms the state-of-the-art methods
on the NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400 datasets in all
15 settings over different views, subjects, setups, and input modalities, and
STF also shows better accuracy on scarce data and dataset shifting settings.

该研究提出了一种基于骨架的动作识别框架，利用空间 - 时间梯度来聚焦相关的空间 - 时间特征，并利用可学习的梯度增强和实例依赖邻接矩阵建立高阶空间 - 时间动态模型，并通过基于梯度的空间 - 时间注意力来指导分类器去关注何时以及何处，以优化堆叠的 STF 模块，该方法在几个数据集上呈现出具有竞争力的结果。