Point cloud sequences are irregular and unordered in the spatial dimension while exhibiting regularities and order in the temporal dimension. Therefore, existing grid based convolutions for conventional video processing cannot be directly applied to spatio-temporal modeling of raw point cloud sequences. In this paper, we propose a point spatio-temporal (PST) convolution to achieve informative representations of point cloud sequences. The proposed PST convolution first disentangles space and time in point cloud sequences. Then, a spatial convolution is employed to capture the local structure of points in the 3D space, and a temporal convolution is used to model the dynamics of the spatial regions along the time dimension. Furthermore, we incorporate the proposed PST convolution into a deep network, namely PSTNet, to extract features of point cloud sequences in a hierarchical manner. Extensive experiments on widely-used 3D action recognition and 4D semantic segmentation datasets demonstrate the effectiveness of PSTNet to model point cloud sequences.

提出了一种名为点时空（PST）卷积的方法，可以用于点云序列的空间-时间建模，该方法首先分离点云序列的空间和时间维度，然后采用空间卷积和时间卷积分别捕捉点在三维空间中的局部结构和在时间维度上的动态变化，并将PST卷积集成到深度网络PSTNet中，以分层方式提取点云序列的特征。实验结果表明，PSTNet方法对于3D动作识别和4D语义分割数据集建模的有效性。

PSTNet: 点云序列上的点空间时卷积