In perception, multiple sensory information is integrated to map visual
information from 2D views onto 3D objects, which is beneficial for
understanding in 3D environments. But in terms of a single 2D view rendered
from different angles, only limited partial information can be provided.The
richness and value of Multi-view 2D information can provide superior
self-supervised signals for 3D objects. In this paper, we propose a novel
self-supervised point cloud representation learning method, MM-Point, which is
driven by intra-modal and inter-modal similarity objectives. The core of
MM-Point lies in the Multi-modal interaction and transmission between 3D
objects and multiple 2D views at the same time. In order to more effectively
simultaneously perform the consistent cross-modal objective of 2D multi-view
information based on contrastive learning, we further propose Multi-MLP and
Multi-level Augmentation strategies. Through carefully designed transformation
strategies, we further learn Multi-level invariance in 2D Multi-views. MM-Point
demonstrates state-of-the-art (SOTA) performance in various downstream tasks.
For instance, it achieves a peak accuracy of 92.4% on the synthetic dataset
ModelNet40, and a top accuracy of 87.8% on the real-world dataset ScanObjectNN,
comparable to fully supervised methods. Additionally, we demonstrate its
effectiveness in tasks such as few-shot classification, 3D part segmentation
and 3D semantic segmentation.

提出一种新颖的自监督点云表示学习方法 MM-Point，通过多模态交互和传输同时处理三维物体和多个二维视图之间的一致跨模态目标，以及通过多个 MLP 和多层次增强策略来更有效地实现二维多视图信息的一致性对比学习，进一步学习了二维多视图的多层次不变性。MM-Point 在各种下游任务中展现出最先进的性能，如在合成数据集 ModelNet40 上达到了 92.4% 的峰值准确率，在真实数据集 ScanObjectNN 上达到了 87.8% 的最高准确率，与全监督方法可媲美。此外，我们还展示了它在少样本分类、三维部分分割和三维语义分割等任务中的有效性。