Manual annotation of large-scale point cloud dataset for varying tasks such
as 3D object classification, segmentation and detection is often laborious
owing to the irregular structure of point clouds. Self-supervised learning,
which operates without any human labeling, is a promising approach to address
this issue. We observe in the real world that humans are capable of mapping the
visual concepts learnt from 2D images to understand the 3D world. Encouraged by
this insight, we propose CrossPoint, a simple cross-modal contrastive learning
approach to learn transferable 3D point cloud representations. It enables a
3D-2D correspondence of objects by maximizing agreement between point clouds
and the corresponding rendered 2D image in the invariant space, while
encouraging invariance to transformations in the point cloud modality. Our
joint training objective combines the feature correspondences within and across
modalities, thus ensembles a rich learning signal from both 3D point cloud and
2D image modalities in a self-supervised fashion. Experimental results show
that our approach outperforms the previous unsupervised learning methods on a
diverse range of downstream tasks including 3D object classification and
segmentation. Further, the ablation studies validate the potency of our
approach for a better point cloud understanding. Code and pretrained models are
available at this http URL

本研究提出了一种名为 CrossPoint 的简单跨模态对比学习方法，通过自监督学习，在不需要人类标注的情况下，实现可转移的三维点云表示，通过最大化点云和相应渲染的二维图像之间的协议，实现了三维物体分类和分割等多项任务，并验证了我们的方法在提高点云理解方面的效果。

CrossPoint: 自监督的跨模态对比学习，用于三维点云理解

CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D  Point Cloud Understanding

Camera relocalization plays a vital role in many robotics and computer vision
tasks, such as global localization, recovery from tracking failure, and loop
closure detection. Recent random forests based methods directly predict 3D
world locations for 2D image locations to guide the camera pose optimization.
During training, each tree greedily splits the samples to minimize the spatial
variance. However, these greedy splits often produce uneven sub-trees in
training or incorrect 2D-3D correspondences in testing. To address these
problems, we propose a sample-balanced objective to encourage equal numbers of
samples in the left and right sub-trees, and a novel backtracking scheme to
remedy the incorrect 2D-3D correspondence predictions. Furthermore, we extend
the regression forests based methods to use local features in both training and
testing stages for outdoor RGB-only applications. Experimental results on
publicly available indoor and outdoor datasets demonstrate the efficacy of our
approach, which shows superior or on-par accuracy with several state-of-the-art
methods.

本文提出了一种基于随机森林的相机定位方法，通过采用样本平衡策略和修正不正确的 2D-3D 对应关系，可以在室内和户外的 RGB 应用场景中实现相对准确的定位效果。