Self-supervised learning (SSL), which aims to learn meaningful prior
representations from unlabeled data, has been proven effective for
label-efficient skeleton-based action understanding. Different from the image
domain, skeleton data possesses sparser spatial structures and diverse
representation forms, with the absence of background clues and the additional
temporal dimension. This presents the new challenges for the pretext task
design of spatial-temporal motion representation learning. Recently, many
endeavors have been made for skeleton-based SSL and remarkable progress has
been achieved. However, a systematic and thorough review is still lacking. In
this paper, we conduct, for the first time, a comprehensive survey on
self-supervised skeleton-based action representation learning, where various
literature is organized according to their pre-training pretext task
methodologies. Following the taxonomy of context-based, generative learning,
and contrastive learning approaches, we make a thorough review and benchmark of
existing works and shed light on the future possible directions. Our
investigation demonstrates that most SSL works rely on the single paradigm,
learning representations of a single level, and are evaluated on the action
recognition task solely, which leaves the generalization power of skeleton SSL
models under-explored. To this end, a novel and effective SSL method for
skeleton is further proposed, which integrates multiple pretext tasks to
jointly learn versatile representations of different granularity, substantially
boosting the generalization capacity for different downstream tasks. Extensive
experiments under three large-scale datasets demonstrate that the proposed
method achieves the superior generalization performance on various downstream
tasks, including recognition, retrieval, detection, and few-shot learning.

本文首次对基于骨架的自监督动作表征学习进行了全面调查和综述，将现有研究根据预训练方法学科进行分类，提出了一种综合多个先验任务的新型骨架自监督学习方法，大大提高了骨架模型在各种下游任务中的泛化性能。在三个大规模数据集上进行的广泛实验证明，该方法在识别、检索、检测和少样本学习等各种下游任务中都取得了卓越的泛化性能。

自主学习的骨架动作表示学习：基准及更多

Self-Supervised Skeleton Action Representation Learning: A Benchmark and  Beyond

Unsupervised pre-training has shown great success in skeleton-based action
understanding recently. Existing works typically train separate
modality-specific models, then integrate the multi-modal information for action
understanding by a late-fusion strategy. Although these approaches have
achieved significant performance, they suffer from the complex yet redundant
multi-stream model designs, each of which is also limited to the fixed input
skeleton modality. To alleviate these issues, in this paper, we propose a
Unified Multimodal Unsupervised Representation Learning framework, called
UmURL, which exploits an efficient early-fusion strategy to jointly encode the
multi-modal features in a single-stream manner. Specifically, instead of
designing separate modality-specific optimization processes for uni-modal
unsupervised learning, we feed different modality inputs into the same stream
with an early-fusion strategy to learn their multi-modal features for reducing
model complexity. To ensure that the fused multi-modal features do not exhibit
modality bias, i.e., being dominated by a certain modality input, we further
propose both intra- and inter-modal consistency learning to guarantee that the
multi-modal features contain the complete semantics of each modal via feature
decomposition and distinct alignment. In this manner, our framework is able to
learn the unified representations of uni-modal or multi-modal skeleton input,
which is flexible to different kinds of modality input for robust action
understanding in practical cases. Extensive experiments conducted on three
large-scale datasets, i.e., NTU-60, NTU-120, and PKU-MMD II, demonstrate that
UmURL is highly efficient, possessing the approximate complexity with the
uni-modal methods, while achieving new state-of-the-art performance across
various downstream task scenarios in skeleton-based action representation
learning.

无监督预训练及早融合策略的统一多模态无监督表示学习框架（UmURL）旨在提取骨架输入的统一表示，通过特征分解和对齐保证多模态特征包含各模态的完整语义，并在各种场景中实现鲁棒动作理解。该框架在三个大规模数据集上取得了新的多种下游任务场景中骨架 - based 动作表示学习的最先进性能。