Pre-training is crucial in 3D-related fields such as autonomous driving where
point cloud annotation is costly and challenging. Many recent studies on point
cloud pre-training, however, have overlooked the issue of incompleteness, where
only a fraction of the points are captured by LiDAR, leading to ambiguity
during the training phase. On the other hand, images offer more comprehensive
information and richer semantics that can bolster point cloud encoders in
addressing the incompleteness issue inherent in point clouds. Yet,
incorporating images into point cloud pre-training presents its own challenges
due to occlusions, potentially causing misalignments between points and pixels.
In this work, we propose PRED, a novel image-assisted pre-training framework
for outdoor point clouds in an occlusion-aware manner. The main ingredient of
our framework is a Birds-Eye-View (BEV) feature map conditioned semantic
rendering, leveraging the semantics of images for supervision through neural
rendering. We further enhance our model's performance by incorporating
point-wise masking with a high mask ratio (95%). Extensive experiments
demonstrate PRED's superiority over prior point cloud pre-training methods,
providing significant improvements on various large-scale datasets for 3D
perception tasks. Codes will be available at this https URL

我们提出了一种新颖的图像辅助预训练框架 PRED，用于户外点云，以解决点云不完整性和遮挡对齐的问题，并通过神经渲染技术利用图像的语义信息进行监督。大规模数据集上的实验证明了 PRED 方法在 3D 感知任务中的显著优势。