Current point-cloud detection methods have difficulty detecting the
open-vocabulary objects in the real world, due to their limited generalization
capability. Moreover, it is extremely laborious and expensive to collect and
fully annotate a point-cloud detection dataset with numerous classes of
objects, leading to the limited classes of existing point-cloud datasets and
hindering the model to learn general representations to achieve open-vocabulary
point-cloud detection. As far as we know, we are the first to study the problem
of open-vocabulary 3D point-cloud detection. Instead of seeking a point-cloud
dataset with full labels, we resort to ImageNet1K to broaden the vocabulary of
the point-cloud detector. We propose OV-3DETIC, an Open-Vocabulary 3D DETector
using Image-level Class supervision. Specifically, we take advantage of two
modalities, the image modality for recognition and the point-cloud modality for
localization, to generate pseudo labels for unseen classes. Then we propose a
novel debiased cross-modal contrastive learning method to transfer the
knowledge from image modality to point-cloud modality during training. Without
hurting the latency during inference, OV-3DETIC makes the point-cloud detector
capable of achieving open-vocabulary detection. Extensive experiments
demonstrate that the proposed OV-3DETIC achieves at least 10.77 % mAP
improvement (absolute value) and 9.56 % mAP improvement (absolute value) by a
wide range of baselines on the SUN-RGBD dataset and ScanNet dataset,
respectively. Besides, we conduct sufficient experiments to shed light on why
the proposed OV-3DETIC works.

OV-3DETIC 是一个开放式词汇的 3D 点云检测器，使用基于图像的分类监督，并利用伪标签和实现跨模态对比学习方法从 ImageNet1K 扩展了其单词表，从而使点云检测器具有实现开放式词汇检测的能力，并实现了至少 10.77% 的 mAP 绝对值改进。