Open-vocabulary object detection (OVD) requires solid modeling of the
region-semantic relationship, which could be learned from massive region-text
pairs. However, such data is limited in practice due to significant annotation
costs. In this work, we propose RTGen to generate scalable open-vocabulary
region-text pairs and demonstrate its capability to boost the performance of
open-vocabulary object detection. RTGen includes both text-to-region and
region-to-text generation processes on scalable image-caption data. The
text-to-region generation is powered by image inpainting, directed by our
proposed scene-aware inpainting guider for overall layout harmony. For
region-to-text generation, we perform multiple region-level image captioning
with various prompts and select the best matching text according to CLIP
similarity. To facilitate detection training on region-text pairs, we also
introduce a localization-aware region-text contrastive loss that learns object
proposals tailored with different localization qualities. Extensive experiments
demonstrate that our RTGen can serve as a scalable, semantically rich, and
effective source for open-vocabulary object detection and continue to improve
the model performance when more data is utilized, delivering superior
performance compared to the existing state-of-the-art methods.

通过生成可扩展的开放词汇区域 - 文本对，RTGen 能够提高开放词汇物体检测的性能，其中文本到区域的生成是通过图像修复以及布局引导来实现的，区域到文本的生成则是通过基于多个提示进行区域级别图像字幕并根据 CLIP 相似性选择最佳匹配文本完成的，并且引入了适应不同定位质量的定位感知区域 - 文本对比损失用于侦测训练。大量实验证明 RTGen 作为可扩展、语义丰富和高效的来源，能够在利用更多数据时持续提高模型性能，并且相较于现有最优方法能够提供卓越性能。