Self-supervised learning (SSL), which aims to learn meaningful prior
representations from unlabeled data, has been proven effective for
label-efficient skeleton-based action understanding. Different from the image
domain, skeleton data possesses sparser spatial structures and diverse
representation forms, with the absence of background clues and the additional
temporal dimension. This presents the new challenges for the pretext task
design of spatial-temporal motion representation learning. Recently, many
endeavors have been made for skeleton-based SSL and remarkable progress has
been achieved. However, a systematic and thorough review is still lacking. In
this paper, we conduct, for the first time, a comprehensive survey on
self-supervised skeleton-based action representation learning, where various
literature is organized according to their pre-training pretext task
methodologies. Following the taxonomy of context-based, generative learning,
and contrastive learning approaches, we make a thorough review and benchmark of
existing works and shed light on the future possible directions. Our
investigation demonstrates that most SSL works rely on the single paradigm,
learning representations of a single level, and are evaluated on the action
recognition task solely, which leaves the generalization power of skeleton SSL
models under-explored. To this end, a novel and effective SSL method for
skeleton is further proposed, which integrates multiple pretext tasks to
jointly learn versatile representations of different granularity, substantially
boosting the generalization capacity for different downstream tasks. Extensive
experiments under three large-scale datasets demonstrate that the proposed
method achieves the superior generalization performance on various downstream
tasks, including recognition, retrieval, detection, and few-shot learning.

本文首次对基于骨架的自监督动作表征学习进行了全面调查和综述，将现有研究根据预训练方法学科进行分类，提出了一种综合多个先验任务的新型骨架自监督学习方法，大大提高了骨架模型在各种下游任务中的泛化性能。在三个大规模数据集上进行的广泛实验证明，该方法在识别、检索、检测和少样本学习等各种下游任务中都取得了卓越的泛化性能。