Existing object detection methods are bounded in a fixed-set vocabulary by
costly labeled data. When dealing with novel categories, the model has to be
retrained with more bounding box annotations. Natural language supervision is
an attractive alternative for its annotation-free attributes and broader object
concepts. However, learning open-vocabulary object detection from language is
challenging since image-text pairs do not contain fine-grained object-language
alignments. Previous solutions rely on either expensive grounding annotations
or distilling classification-oriented vision models. In this paper, we propose
a novel open-vocabulary object detection framework directly learning from
image-text pair data. We formulate object-language alignment as a set matching
problem between a set of image region features and a set of word embeddings. It
enables us to train an open-vocabulary object detector on image-text pairs in a
much simple and effective way. Extensive experiments on two benchmark datasets,
COCO and LVIS, demonstrate our superior performance over the competing
approaches on novel categories, e.g. achieving 32.0% mAP on COCO and 21.7% mask
mAP on LVIS. Code is available at: this https URL

提出了一种使用自然语言监督学习从配对的图像文本数据中直接学习的新型开放词汇目标检测框架，通过将对象 - 语言对齐形式化为一个图像区域特征集合和一个词向量集合之间的集合匹配问题，可以更简单和有效地在图像 - 文本对上训练开放词汇对象探测器，并在两个基准数据集上进行了广泛的实验，在 LVIS 等新类别上实现了 32.0％的 mAP 和 21.7％的掩膜 mAP 等卓越性能。