In this paper, we focus on unsupervised representation learning for
skeleton-based action recognition. Existing approaches usually learn action
representations by sequential prediction but they suffer from the inability to
fully learn semantic information. To address this limitation, we propose a
novel framework named Prototypical Contrast and Reverse Prediction (PCRP),
which not only creates reverse sequential prediction to learn low-level
information (e.g., body posture at every frame) and high-level pattern (e.g.,
motion order), but also devises action prototypes to implicitly encode semantic
similarity shared among sequences. In general, we regard action prototypes as
latent variables and formulate PCRP as an expectation-maximization task.
Specifically, PCRP iteratively runs (1) E-step as determining the distribution
of prototypes by clustering action encoding from the encoder, and (2) M-step as
optimizing the encoder by minimizing the proposed ProtoMAE loss, which helps
simultaneously pull the action encoding closer to its assigned prototype and
perform reverse prediction task. Extensive experiments on N-UCLA, NTU 60, and
NTU 120 dataset present that PCRP outperforms state-of-the-art unsupervised
methods and even achieves superior performance over some of supervised methods.
Codes are available at this https URL

PCRP 网络提出了一种结合了原型聚类和反向预测的无监督表示学习框架，利用潜在变量隐式编码序列之间的语义相似性，并通过期望最大化算法进行优化，实现了在骨骼动作识别中超越监督学习方法的效果。