Open World Object Detection (OWOD) is a novel computer vision task with a
considerable challenge, bridging the gap between classic object detection (OD)
benchmarks and real-world object detection. In addition to detecting and
classifying seen/known objects, OWOD algorithms are expected to localize all
potential unseen/unknown objects and incrementally learn them. The large
pre-trained vision-language grounding models (VLM,eg, GLIP) have rich knowledge
about the open world, but are limited by text prompts and cannot localize
indescribable objects. However, there are many detection scenarios which
pre-defined language descriptions are unavailable during inference. In this
paper, we attempt to specialize the VLM model for OWOD task by distilling its
open-world knowledge into a language-agnostic detector. Surprisingly, we
observe that the combination of a simple knowledge distillation approach and
the automatic pseudo-labeling mechanism in OWOD can achieve better performance
for unknown object detection, even with a small amount of data. Unfortunately,
knowledge distillation for unknown objects severely affects the learning of
detectors with conventional structures for known objects, leading to
catastrophic forgetting. To alleviate these problems, we propose the
down-weight loss function for knowledge distillation from vision-language to
single vision modality. Meanwhile, we decouple the learning of localization and
recognition to reduce the impact of category interactions of known and unknown
objects on the localization learning process. Comprehensive experiments
performed on MS-COCO and PASCAL VOC demonstrate the effectiveness of our
methods.

通过蒸馏视觉 - 语言模型中的开放世界知识，并提出一种权重降低的损失函数以及通过分离定位和识别来减少已知和未知对象的相互影响，我们成功地改进了未知对象的检测性能。