Traditional object detection methods operate under the closed-set assumption, where models can only detect a fixed number of objects predefined in the training set. Recent works on open vocabulary object detection (OVD) enable the detection of objects defined by an unbounded vocabulary, which reduces the cost of training models for specific tasks. However, OVD heavily relies on accurate prompts provided by an ''oracle'', which limits their use in critical applications such as driving scene perception. OVD models tend to misclassify near-out-of-distribution (NOOD) objects that have similar semantics to known classes, and ignore far-out-of-distribution (FOOD) objects. To address theses limitations, we propose a framework that enables OVD models to operate in open world settings, by identifying and incrementally learning novel objects. To detect FOOD objects, we propose Open World Embedding Learning (OWEL) and introduce the concept of Pseudo Unknown Embedding which infers the location of unknown classes in a continuous semantic space based on the information of known classes. We also propose Multi-Scale Contrastive Anchor Learning (MSCAL), which enables the identification of misclassified unknown objects by promoting the intra-class consistency of object embeddings at different scales. The proposed method achieves state-of-the-art performance in common open world object detection and autonomous driving benchmarks.

本研究解决了传统对象检测方法在固定集合假设下的局限性，尤其是在开放词汇对象检测中未能有效处理近似已知类别和远离已知类别的新对象的问题。提出了一种新框架，使OVD模型能够在开放世界环境中识别并逐步学习新对象，通过开放世界嵌入学习和多尺度对比锚学习等方法，显著提升了对象检测的性能，尤其在自动驾驶场景中展现出更高的准确性。

从开放词汇到开放世界：教导视觉语言模型检测新颖对象