Self-supervised learning has proved effective for skeleton-based human action
understanding, which is an important yet challenging topic. Previous works
mainly rely on contrastive learning or masked motion modeling paradigm to model
the skeleton relations. However, the sequence-level and joint-level
representation learning cannot be effectively and simultaneously handled by
these methods. As a result, the learned representations fail to generalize to
different downstream tasks. Moreover, combining these two paradigms in a naive
manner leaves the synergy between them untapped and can lead to interference in
training. To address these problems, we propose Prompted Contrast with Masked
Motion Modeling, PCM$^{\rm 3}$, for versatile 3D action representation
learning. Our method integrates the contrastive learning and masked prediction
tasks in a mutually beneficial manner, which substantially boosts the
generalization capacity for various downstream tasks. Specifically, masked
prediction provides novel training views for contrastive learning, which in
turn guides the masked prediction training with high-level semantic
information. Moreover, we propose a dual-prompted multi-task pretraining
strategy, which further improves model representations by reducing the
interference caused by learning the two different pretext tasks. Extensive
experiments on five downstream tasks under three large-scale datasets are
conducted, demonstrating the superior generalization capacity of PCM$^{\rm 3}$
compared to the state-of-the-art works. Our project is publicly available at:
this https URL .

我们提出了 Prompted Contrast with Masked Motion Modeling (PCM3) 方法，通过互惠方式集成了对比学习和遮挡预测任务，极大地提高了在不同下游任务中的泛化能力。通过减少学习两个不同训练任务造成的干扰，并通过掩模预测为对比学习提供了新视角，指导了高级语义信息的预测训练。与现有的工作相比，在三个大规模数据集上进行了大量实验证明了 PCM3 方法的卓越泛化能力。