Open-vocabulary object detection (OVOD) aims to recognize novel objects whose categories are not included in training set. In order to classify these unseen classes during training, many OVOD frameworks leverage the zero-shot capability of largely pretrained vision and language models, such as CLIP. To further improve generalization on the unseen novel classes, several approaches proposed to additionally train with pseudo region labeling on the external data sources that contain a substantial number of novel category labels beyond the existing training data. Albeit its simplicity, these pseudo-labeling methods still exhibit limited improvement with regard to the genuine novel classes that were not pseudo-labeled. In this paper, we present a novel, yet simple technique that helps generalization on the overall distribution of novel classes. Inspired by our observation that numerous novel classes reside within the convex hull constructed by the base (seen) classes in the CLIP embedding space, we propose to synthesize proxy-novel classes approximating novel classes via linear mixup between a pair of base classes. By training our detector with these synthetic proxy-novel classes, we effectively explore the embedding space of novel classes. The experimental results on various OVOD benchmarks such as LVIS and COCO demonstrate superior performance on novel classes compared to the other state-of-the-art methods.

通过利用预训练的视觉和语言模型（如CLIP）的零样本能力，结合伪区域标注的外部数据源，提出了一种在CLIP嵌入空间中通过线性混合虚拟生成接近新颖类的代理新颖类的新颖而简单的技术，该技术能够在整体新颖类分布上改进开放词汇目标检测模型的泛化能力，并在LVIS和COCO等各种开放词汇目标检测基准上表现出优越的新颖类分类性能。

ProxyDet：通过类别混合合成代理新类别，用于开放词汇的目标检测