Open-Vocabulary Object Detection (OVOD) aims to detect novel objects beyond a
given set of base categories on which the detection model is trained. Recent
OVOD methods focus on adapting the image-level pre-trained vision-language
models (VLMs), such as CLIP, to a region-level object detection task via, eg.,
region-level knowledge distillation, regional prompt learning, or region-text
pre-training, to expand the detection vocabulary. These methods have
demonstrated remarkable performance in recognizing regional visual concepts,
but they are weak in exploiting the VLMs' powerful global scene understanding
ability learned from the billion-scale image-level text descriptions. This
limits their capability in detecting hard objects of small, blurred, or
occluded appearance from novel/base categories, whose detection heavily relies
on contextual information. To address this, we propose a novel approach, namely
Simple Image-level Classification for Context-Aware Detection Scoring
(SIC-CADS), to leverage the superior global knowledge yielded from CLIP for
complementing the current OVOD models from a global perspective. The core of
SIC-CADS is a multi-modal multi-label recognition (MLR) module that learns the
object co-occurrence-based contextual information from CLIP to recognize all
possible object categories in the scene. These image-level MLR scores can then
be utilized to refine the instance-level detection scores of the current OVOD
models in detecting those hard objects. This is verified by extensive empirical
results on two popular benchmarks, OV-LVIS and OV-COCO, which show that
SIC-CADS achieves significant and consistent improvement when combined with
different types of OVOD models. Further, SIC-CADS also improves the
cross-dataset generalization ability on Objects365 and OpenImages. The code is
available at this https URL

利用简单的图像级别分类方法（Simple Image-level Classification）结合上下文感知的检测得分（Context-Aware Detection Scoring）模块，从全局视角利用 CLIP 模型的全局知识来优化当前 OVOD 模型，以便检测小型、模糊或遮挡的新颖 / 基础类别的难以检测的目标。