Open-world detection poses significant challenges, as it requires the
detection of any object using either object class labels or free-form texts.
Existing related works often use large-scale manual annotated caption datasets
for training, which are extremely expensive to collect. Instead, we propose to
transfer knowledge from vision-language models (VLMs) to enrich the
open-vocabulary descriptions automatically. Specifically, we bootstrap dense
synthetic captions using pre-trained VLMs to provide rich descriptions on
different regions in images, and incorporate these captions to train a novel
detector that generalizes to novel concepts. To mitigate the noise caused by
hallucination in synthetic captions, we also propose a novel hyperbolic
vision-language learning approach to impose a hierarchy between visual and
caption embeddings. We call our detector ``HyperLearner''. We conduct extensive
experiments on a wide variety of open-world detection benchmarks (COCO, LVIS,
Object Detection in the Wild, RefCOCO) and our results show that our model
consistently outperforms existing state-of-the-art methods, such as GLIP,
GLIPv2 and Grounding DINO, when using the same backbone.

通过将视觉语言模型 (VLMs) 的知识传递进行启发式操作，从而丰富开放词汇的图像描述。同时，采用新的超几何视觉语言学习方法对合成字幕产生的噪音进行缓解，实现了视觉与字幕嵌入之间的层次关系。在各种开放世界检测基准测试 (COCO、LVIS、Object Detection in the Wild、RefCOCO) 上进行了广泛实验，结果表明我们的模型在使用相同的主干网络时，始终优于现有的 GLIP、GLIPv2 和 Grounding DINO 等最先进方法。