State-of-the-art 3D object detectors are often trained on massive labeled
datasets. However, annotating 3D bounding boxes remains prohibitively expensive
and time-consuming, particularly for LiDAR. Instead, recent works demonstrate
that self-supervised pre-training with unlabeled data can improve detection
accuracy with limited labels. Contemporary methods adapt best-practices for
self-supervised learning from the image domain to point clouds (such as
contrastive learning). However, publicly available 3D datasets are considerably
smaller and less diverse than those used for image-based self-supervised
learning, limiting their effectiveness. We do note, however, that such data is
naturally collected in a multimodal fashion, often paired with images. Rather
than pre-training with only self-supervised objectives, we argue that it is
better to bootstrap point cloud representations using image-based foundation
models trained on internet-scale image data. Specifically, we propose a
shelf-supervised approach (e.g. supervised with off-the-shelf image foundation
models) for generating zero-shot 3D bounding boxes from paired RGB and LiDAR
data. Pre-training 3D detectors with such pseudo-labels yields significantly
better semi-supervised detection accuracy than prior self-supervised pretext
tasks. Importantly, we show that image-based shelf-supervision is helpful for
training LiDAR-only and multi-modal (RGB + LiDAR) detectors. We demonstrate the
effectiveness of our approach on nuScenes and WOD, significantly improving over
prior work in limited data settings.

通过使用与图像相关的 shelf-supervision 模型对配对的 RGB 和 LiDAR 数据进行无监督预训练，我们提出了一种能够生成零样本 3D 边界框的点云表示方法，这种方法在半监督检测中能够显著提高检测准确性，尤其适用于 LiDAR-only 和多模态 (RGB + LiDAR) 检测。我们在 nuScenes 和 WOD 上展示了我们方法的有效性，并在有限数据环境中明显改进了之前的工作。