We present a new self-supervised paradigm on point cloud sequence
understanding. Inspired by the discriminative and generative self-supervised
methods, we design two tasks, namely point cloud sequence based Contrastive
Prediction and Reconstruction (CPR), to collaboratively learn more
comprehensive spatiotemporal representations. Specifically, dense point cloud
segments are first input into an encoder to extract embeddings. All but the
last ones are then aggregated by a context-aware autoregressor to make
predictions for the last target segment. Towards the goal of modeling
multi-granularity structures, local and global contrastive learning are
performed between predictions and targets. To further improve the
generalization of representations, the predictions are also utilized to
reconstruct raw point cloud sequences by a decoder, where point cloud
colorization is employed to discriminate against different frames. By combining
classic contrast and reconstruction paradigms, it makes the learned
representations with both global discrimination and local perception. We
conduct experiments on four point cloud sequence benchmarks, and report the
results on action recognition and gesture recognition under multiple
experimental settings. The performances are comparable with supervised methods
and show powerful transferability.

本文提出了一种新的自监督范式来处理点云序列理解问题，设计了两个任务：基于点云序列的对比预测和重建（CPR）来协同学习更全面的时空表示。通过组合传统的对比和重建范例，使得学习到的表示在全局判别和局部感知方面都更加优越。

用于动态点云自监督学习的对比预测自编码器

Contrastive Predictive Autoencoders for Dynamic Point Cloud  Self-Supervised Learning

The point cloud sequence of 3D human actions consists of a set of ordered
point cloud frames. Compared to static point clouds, point cloud sequences have
huge data sizes proportional to the time dimension. Therefore, developing an
efficient and lightweight point cloud sequence model is pivotal for 3D action
recognition. In this paper, we propose a strong frame-level parallel point
cloud sequence network referred to as SequentialPointNet for 3D action
recognition. The key to our approach is to divide the main modeling operations
into frame-level units executed in parallel, which greatly improves the
efficiency of modeling point cloud sequences.Moreover, we propose to flatten
the point cloud sequence into a new point data type named hyperpoint sequence
that preserves the complete spatial structure of each frame. Then, a novel
Hyperpoint-Mixer module is introduced to mix intra-frame spatial features and
inter-frame temporal features of the hyperpoint sequence. By doing so,
SequentialPointNet maximizes the appearance encoding ability and extracts
sufficient motion information for effective human action recognition. Extensive
experiments show that SequentialPointNet achieves up to 10X faster than
existing point cloud sequence models. Additionally, our SequentialPointNet
surpasses state-of-the-art approaches for human action recognition on both
large-scale datasets (i.e., NTU RGB+D 60 and NTU RGB+D 120) and small-scale
datasets (i.e., MSR Action3D and UTD-MHAD).

本研究提出了一个有效而轻量化的点云序列模型，能够在 3D 动作识别方面实现较高的准确性和效率，主要包括点云序列数据扁平化和超点序列混合等方法。实验证明，该模型在速度和效果方面均优于现有模型。