3D point cloud understanding has made great progress in recent years.
However, one major bottleneck is the scarcity of annotated real datasets,
especially compared to 2D object detection tasks, since a large amount of labor
is involved in annotating the real scans of a scene. A promising solution to
this problem is to make better use of the synthetic dataset, which consists of
CAD object models, to boost the learning on real datasets. This can be achieved
by the pre-training and fine-tuning procedure. However, recent work on 3D
pre-training exhibits failure when transfer features learned on synthetic
objects to other real-world applications. In this work, we put forward a new
method called RandomRooms to accomplish this objective. In particular, we
propose to generate random layouts of a scene by making use of the objects in
the synthetic CAD dataset and learn the 3D scene representation by applying
object-level contrastive learning on two random scenes generated from the same
set of synthetic objects. The model pre-trained in this way can serve as a
better initialization when later fine-tuning on the 3D object detection task.
Empirically, we show consistent improvement in downstream 3D detection tasks on
several base models, especially when less training data are used, which
strongly demonstrates the effectiveness and generalization of our method.
Benefiting from the rich semantic knowledge and diverse objects from synthetic
data, our method establishes the new state-of-the-art on widely-used 3D
detection benchmarks ScanNetV2 and SUN RGB-D. We expect our attempt to provide
a new perspective for bridging object and scene-level 3D understanding.

该研究提出一种名为 RandomRooms 的新方法，利用合成 CAD 数据集中的对象生成场景的随机布局并通过对象级对比学习来学习 3D 场景表示，并表明该模型可用于提高 3D 检测任务的性能和泛化能力，从而在 ScanNetV2 和 SUN RGB-D 等流行的 3D 检测基准上取得新的最优结果。