Skeleton-based action recognition has recently made significant progress.
However, data imbalance is still a great challenge in real-world scenarios. The
performance of current action recognition algorithms declines sharply when
training data suffers from heavy class imbalance. The imbalanced data actually
degrades the representations learned by these methods and becomes the
bottleneck for action recognition. How to learn unbiased representations from
imbalanced action data is the key to long-tailed action recognition. In this
paper, we propose a novel balanced representation learning method to address
the long-tailed problem in action recognition. Firstly, a spatial-temporal
action exploration strategy is presented to expand the sample space
effectively, generating more valuable samples in a rebalanced manner. Secondly,
we design a detached action-aware learning schedule to further mitigate the
bias in the representation space. The schedule detaches the representation
learning of tail classes from training and proposes an action-aware loss to
impose more effective constraints. Additionally, a skip-modal representation is
proposed to provide complementary structural information. The proposed method
is validated on four skeleton datasets, NTU RGB+D 60, NTU RGB+D 120, NW-UCLA,
and Kinetics. It not only achieves consistently large improvement compared to
the state-of-the-art (SOTA) methods, but also demonstrates a superior
generalization capacity through extensive experiments. Our code is available at
this https URL

骨架动作识别在最近取得了显著进展，然而，在现实场景中存在数据不平衡是一个巨大的挑战，当前动作识别算法的性能在训练数据中存在严重类别不平衡时急剧下降，如何从不平衡的动作数据中学习无偏的表示是解决长尾动作识别问题的关键，本文提出了一种新颖的平衡表示学习方法，来解决动作识别中的长尾问题，首先，提出了一种空间 - 时间动作探索策略，以重新平衡的方式扩展样本空间，生成更有价值的样本，其次，设计了一种分离的动作感知学习计划，以进一步减轻表示空间中的偏差，该计划从训练中分离了尾类的表示学习，并提出了一种动作感知损失以施加更有效的约束，此外，提出了一种跳过模态表示，以提供互补的结构信息，在四个骨架数据集（NTU RGB+D 60，NTU RGB+D 120，NW-UCLA 和 Kinetics）上验证了所提出的方法，它不仅相对于最先进的方法取得了一致的大幅改进，还通过大量实验证明了其卓越的泛化能力。我们的代码可以在此 https URL 获取。