We present a new open-vocabulary detection framework. Our framework uses both
image-level labels and detailed detection annotations when available. Our
framework proceeds in three steps. We first train a language-conditioned object
detector on fully-supervised detection data. This detector gets to see the
presence or absence of ground truth classes during training, and conditions
prediction on the set of present classes. We use this detector to pseudo-label
images with image-level labels. Our detector provides much more accurate
pseudo-labels than prior approaches with its conditioning mechanism. Finally,
we train an unconditioned open-vocabulary detector on the pseudo-annotated
images. The resulting detector, named DECOLA, shows strong zero-shot
performance in open-vocabulary LVIS benchmark as well as direct zero-shot
transfer benchmarks on LVIS, COCO, Object365, and OpenImages. DECOLA
outperforms the prior arts by 17.1 AP-rare and 9.4 mAP on zero-shot LVIS
benchmark. DECOLA achieves state-of-the-art results in various model sizes,
architectures, and datasets by only training on open-sourced data and
academic-scale computing. Code is available at
this https URL

我们提出了一个新的开放词汇检测框架 DECOLA，该框架使用语言条件下的物体检测器和伪标签来实现零样本性能。