The pre-trained point cloud model based on Masked Point Modeling (MPM) has
exhibited substantial improvements across various tasks. However, these models
heavily rely on the Transformer, leading to quadratic complexity and limited
decoder, hindering their practice application. To address this limitation, we
first conduct a comprehensive analysis of existing Transformer-based MPM,
emphasizing the idea that redundancy reduction is crucial for point cloud
analysis. To this end, we propose a Locally constrained Compact point cloud
Model (LCM) consisting of a locally constrained compact encoder and a locally
constrained Mamba-based decoder. Our encoder replaces self-attention with our
local aggregation layers to achieve an elegant balance between performance and
efficiency. Considering the varying information density between masked and
unmasked patches in the decoder inputs of MPM, we introduce a locally
constrained Mamba-based decoder. This decoder ensures linear complexity while
maximizing the perception of point cloud geometry information from unmasked
patches with higher information density. Extensive experimental results show
that our compact model significantly surpasses existing Transformer-based
models in both performance and efficiency, especially our LCM-based Point-MAE
model, compared to the Transformer-based model, achieved an improvement of
2.24%, 0.87%, and 0.94% in performance on the three variants of ScanObjectNN
while reducing parameters by 88% and computation by 73%.

提出了一种基于局部约束紧凑编码器和局部约束 Mamba 解码器的局部约束紧凑点云模型 (LCM)，它在性能和效率方面显著超越了现有的基于 Transformer 的模型，尤其是在 ScanObjectNN 的三个变体上，相比于基于 Transformer 的模型，在性能上改进了 2.24%、0.87% 和 0.94%，同时减少了 88% 的参数和 73% 的计算。

LCM：局部约束紧凑点云模型用于点云建模

LCM: Locally Constrained Compact Point Cloud Model for Masked Point  Modeling

Masked point modeling has become a promising scheme of self-supervised
pre-training for point clouds. Existing methods reconstruct either the original
points or related features as the objective of pre-training. However,
considering the diversity of downstream tasks, it is necessary for the model to
have both low- and high-level representation modeling capabilities to capture
geometric details and semantic contexts during pre-training. To this end,
M$^3$CS is proposed to enable the model with the above abilities. Specifically,
with masked point cloud as input, M$^3$CS introduces two decoders to predict
masked representations and the original points simultaneously. While an extra
decoder doubles parameters for the decoding process and may lead to
overfitting, we propose siamese decoders to keep the amount of learnable
parameters unchanged. Further, we propose an online codebook projecting
continuous tokens into discrete ones before reconstructing masked points. In
such way, we can enforce the decoder to take effect through the combinations of
tokens rather than remembering each token. Comprehensive experiments show that
M$^3$CS achieves superior performance at both classification and segmentation
tasks, outperforming existing methods.

M^3CS 是一种掩码点云建模方案，通过引入 siamese 解码器和在线码书投影，它能够同时预测掩码表示和原始点，从而在自我监督预训练中获得低级和高级表示建模能力，并在分类和分割任务中显示出优越性能。