Existing perception methods for autonomous driving fall short of recognizing
unknown entities not covered in the training data. Open-vocabulary methods
offer promising capabilities in detecting any object but are limited by
user-specified queries representing target classes. We propose AutoVoc3D, a
framework for automatic object class recognition and open-ended segmentation.
Evaluation on nuScenes showcases AutoVoc3D's ability to generate precise
semantic classes and accurate point-wise segmentation. Moreover, we introduce
Text-Point Semantic Similarity, a new metric to assess the semantic similarity
between text and point cloud without eliminating novel classes.

自动驾驶中现有的感知方法无法识别训练数据中未覆盖的未知实体。开放词汇方法能够检测任何目标，但受限于用户指定的查询代表目标类别。我们提出了 AutoVoc3D，一种用于自动目标类别识别和开放式分割的框架。在 nuScenes 中的评估显示了 AutoVoc3D 生成精确的语义类别和准确的逐点分割的能力。此外，我们引入了一种新的度量方法 Text-Point 语义相似性，用于评估文本和点云之间的语义相似性，而不排除新的类别。

激光雷达点的自动词汇分割

Auto-Vocabulary Segmentation for LiDAR Points

Recently, a few open-vocabulary methods have been proposed by employing a
unified architecture to tackle generic segmentation and detection tasks.
However, their performance still lags behind the task-specific models due to
the conflict between different tasks, and their open-vocabulary capability is
limited due to the inadequate use of CLIP. To address these challenges, we
present a universal transformer-based framework, abbreviated as OpenSD, which
utilizes the same architecture and network parameters to handle open-vocabulary
segmentation and detection tasks. First, we introduce a decoder decoupled
learning strategy to alleviate the semantic conflict between thing and staff
categories so that each individual task can be learned more effectively under
the same framework. Second, to better leverage CLIP for end-to-end segmentation
and detection, we propose dual classifiers to handle the in-vocabulary domain
and out-of-vocabulary domain, respectively. The text encoder is further trained
to be region-aware for both thing and stuff categories through decoupled prompt
learning, enabling them to filter out duplicated and low-quality predictions,
which is important to end-to-end segmentation and detection. Extensive
experiments are conducted on multiple datasets under various circumstances. The
results demonstrate that OpenSD outperforms state-of-the-art open-vocabulary
segmentation and detection methods in both closed- and open-vocabulary
settings. Code is available at this https URL

我们提出了一个名为 OpenSD 的通用 Transformer 基础框架，利用相同的架构和网络参数来处理开放词汇分割和检测任务。OpenSD 在封闭和开放词汇设置下，优于最先进的开放词汇分割和检测方法。