Action recognition with skeleton data has recently attracted much attention in computer vision. Previous studies are mostly based on fixed skeleton graphs, only capturing local physical dependencies among joints, which may miss implicit joint correlations. To capture richer dependencies, we introduce an encoder-decoder structure, called A-link inference module, to capture action-specific latent dependencies, i.e. actional links, directly from actions. We also extend the existing skeleton graphs to represent higher-order dependencies, i.e. structural links. Combing the two types of links into a generalized skeleton graph, we further propose the actional-structural graph convolution network (AS-GCN), which stacks actional-structural graph convolution and temporal convolution as a basic building block, to learn both spatial and temporal features for action recognition. A future pose prediction head is added in parallel to the recognition head to help capture more detailed action patterns through self-supervision. We validate AS-GCN in action recognition using two skeleton data sets, NTU-RGB+D and Kinetics. The proposed AS-GCN achieves consistently large improvement compared to the state-of-the-art methods. As a side product, AS-GCN also shows promising results for future pose prediction.

提出了一种基于编码器-解码器结构的 A-link 推断模块来捕捉动作特定的潜在依赖关系，即动作链接，并将其与现有的骨骼图扩展以表示更高阶依赖关系，即结构链接，进而将两种类型的链接组合成广义骨骼图卷积网络（AS-GCN），用于动作识别，并在 NTU-RGB + D 和 Kinetics 两个骨骼数据集上进行验证，并通过与现有方法的比较实现了一致的显著改进。同时，AS-GCN 在未来姿态预测方面也表现出了良好的结果。

基于动作骨架的行为识别的动作-构造图卷积网络