Recent open-vocabulary detection methods aim to detect novel objects by
distilling knowledge from vision-language models (VLMs) trained on a vast
amount of image-text pairs. To improve the effectiveness of these methods,
researchers have utilized datasets with a large vocabulary that contains a
large number of object classes, under the assumption that such data will enable
models to extract comprehensive knowledge on the relationships between various
objects and better generalize to unseen object classes. In this study, we argue
that more fine-grained labels are necessary to extract richer knowledge about
novel objects, including object attributes and relationships, in addition to
their names. To address this challenge, we propose a simple and effective
method named Pseudo Caption Labeling (PCL), which utilizes an image captioning
model to generate captions that describe object instances from diverse
perspectives. The resulting pseudo caption labels offer dense samples for
knowledge distillation. On the LVIS benchmark, our best model trained on the
de-duplicated VisualGenome dataset achieves an AP of 34.5 and an APr of 30.6,
comparable to the state-of-the-art performance. PCL's simplicity and
flexibility are other notable features, as it is a straightforward
pre-processing technique that can be used with any image captioning model
without imposing any restrictions on model architecture or training process.

该研究提出了一种名为 Pseudo Caption Labeling（PCL）的简单而有效的方法，利用图像字幕模型生成对不同角度目标实例的描述，通过这些大量的数据样本进行知识提炼，以丰富目标的属性和关系等细节，从而提高模型的性能，实验表明该方法可以与任何图像字幕模型一起使用，不需要对模型架构或训练流程进行任何限制。