With the overwhelming trend of mask image modeling led by MAE, generative
pre-training has shown a remarkable potential to boost the performance of
fundamental models in 2D vision. However, in 3D vision, the over-reliance on
Transformer-based backbones and the unordered nature of point clouds have
restricted the further development of generative pre-training. In this paper,
we propose a novel 3D-to-2D generative pre-training method that is adaptable to
any point cloud model. We propose to generate view images from different
instructed poses via the cross-attention mechanism as the pre-training scheme.
Generating view images has more precise supervision than its point cloud
counterpart, thus assisting 3D backbones to have a finer comprehension of the
geometrical structure and stereoscopic relations of the point cloud.
Experimental results have proved the superiority of our proposed 3D-to-2D
generative pre-training over previous pre-training methods. Our method is also
effective in boosting the performance of architecture-oriented approaches,
achieving state-of-the-art performance when fine-tuning on ScanObjectNN
classification and ShapeNetPart segmentation tasks. Code is available at
this https URL

我们提出了一种新颖的适用于任何点云模型的 3D 到 2D 生成预训练方法，通过交叉注意机制生成来自不同指示姿势的视图图像作为预训练方案，以精确监督帮助 3D 主干更好地理解点云的几何结构和立体关系，实验结果证明了我们提出的方法在提升架构导向方法性能方面的优越性，并在 Fine-tuning ScanObjectNN 分类和 ShapeNetPart 分割任务时实现了最先进的性能。