Open-vocabulary 3D Object Detection (OV-3DDet) addresses the detection of
objects from an arbitrary list of novel categories in 3D scenes, which remains
a very challenging problem. In this work, we propose CoDAv2, a unified
framework designed to innovatively tackle both the localization and
classification of novel 3D objects, under the condition of limited base
categories. For localization, the proposed 3D Novel Object Discovery (3D-NOD)
strategy utilizes 3D geometries and 2D open-vocabulary semantic priors to
discover pseudo labels for novel objects during training. 3D-NOD is further
extended with an Enrichment strategy that significantly enriches the novel
object distribution in the training scenes, and then enhances the model's
ability to localize more novel objects. The 3D-NOD with Enrichment is termed
3D-NODE. For classification, the Discovery-driven Cross-modal Alignment (DCMA)
module aligns features from 3D point clouds and 2D/textual modalities,
employing both class-agnostic and class-specific alignments that are
iteratively refined to handle the expanding vocabulary of objects. Besides, 2D
box guidance boosts the classification accuracy against complex background
noises, which is coined as Box-DCMA. Extensive evaluation demonstrates the
superiority of CoDAv2. CoDAv2 outperforms the best-performing method by a large
margin (AP_Novel of 9.17 vs. 3.61 on SUN-RGBD and 9.12 vs. 3.74 on ScanNetv2).
Source code and pre-trained models are available at the GitHub project page.

CoDAv2 是一个统一的框架，用于创新性地解决有限基本类别条件下的新型 3D 物体的定位和分类问题，其中使用 3D 几何和 2D 开放词汇语义先验来发现训练期间新型物体的伪标签以及通过 3D 点云和 2D / 文本模态之间的特征对齐来提升分类准确性。