Open vocabulary object detection (OVD) aims at seeking an optimal object
detector capable of recognizing objects from both base and novel categories.
Recent advances leverage knowledge distillation to transfer insightful
knowledge from pre-trained large-scale vision-language models to the task of
object detection, significantly generalizing the powerful capabilities of the
detector to identify more unknown object categories. However, these methods
face significant challenges in background interpretation and model overfitting
and thus often result in the loss of crucial background knowledge, giving rise
to sub-optimal inference performance of the detector. To mitigate these issues,
we present a novel OVD framework termed LBP to propose learning background
prompts to harness explored implicit background knowledge, thus enhancing the
detection performance w.r.t. base and novel categories. Specifically, we devise
three modules: Background Category-specific Prompt, Background Object
Discovery, and Inference Probability Rectification, to empower the detector to
discover, represent, and leverage implicit object knowledge explored from
background proposals. Evaluation on two benchmark datasets, OV-COCO and
OV-LVIS, demonstrates the superiority of our proposed method over existing
state-of-the-art approaches in handling the OVD tasks.

提出了一种新颖的开放式词汇目标检测（OVD）框架，通过学习背景提示来增强检测性能，涉及背景解释、模型过拟合和检测性能等方面的问题。在 OV-COCO 和 OV-LVIS 两个基准数据集上的评估结果表明，我们提出的方法在处理 OVD 任务时优于现有的最先进方法。

利用学习背景提示来发现开放词汇对象检测的隐含知识

Learning Background Prompts to Discover Implicit Knowledge for Open  Vocabulary Object Detection

Open vocabulary object detection has been greatly advanced by the recent
development of vision-language pretrained model, which helps recognize novel
objects with only semantic categories. The prior works mainly focus on
knowledge transferring to the object proposal classification and employ
class-agnostic box and mask prediction. In this work, we propose CondHead, a
principled dynamic network design to better generalize the box regression and
mask segmentation for open vocabulary setting. The core idea is to
conditionally parameterize the network heads on semantic embedding and thus the
model is guided with class-specific knowledge to better detect novel
categories. Specifically, CondHead is composed of two streams of network heads,
the dynamically aggregated head and the dynamically generated head. The former
is instantiated with a set of static heads that are conditionally aggregated,
these heads are optimized as experts and are expected to learn sophisticated
prediction. The latter is instantiated with dynamically generated parameters
and encodes general class-specific information. With such a conditional design,
the detection model is bridged by the semantic embedding to offer strongly
generalizable class-wise box and mask prediction. Our method brings significant
improvement to the state-of-the-art open vocabulary object detection methods
with very minor overhead, e.g., it surpasses a RegionClip model by 3.0
detection AP on novel categories, with only 1.1% more computation.

本文提出 CondHead 作为一种动态网络设计，通过对语义嵌入条件参数化来指导模型根据类特定知识更好地检测新类别，从而使检测模型通过语义嵌入提供强大的可推广类别框和掩膜预测，并在非常小的开销下显著改善了开放词汇的目标检测方法。