In this paper, we introduce a novel paradigm to enhance the ability of object
detector, e.g., expanding categories or improving detection performance, by
training on synthetic dataset generated from diffusion models. Specifically, we
integrate an instance-level grounding head into a pre-trained, generative
diffusion model, to augment it with the ability of localising arbitrary
instances in the generated images. The grounding head is trained to align the
text embedding of category names with the regional visual feature of the
diffusion model, using supervision from an off-the-shelf object detector, and a
novel self-training scheme on (novel) categories not covered by the detector.
This enhanced version of diffusion model, termed as InstaGen, can serve as a
data synthesizer for object detection. We conduct thorough experiments to show
that, object detector can be enhanced while training on the synthetic dataset
from InstaGen, demonstrating superior performance over existing
state-of-the-art methods in open-vocabulary (+4.5 AP) and data-sparse (+1.2 to
5.2 AP) scenarios.

通过在扩展类别或提高检测性能方面训练基于扩散模型生成的合成数据集，我们在本文中引入了一种新的范例来增强目标检测器的能力。具体而言，我们将实例级别的定位引导器集成到预训练的生成扩散模型中，使其具备定位生成图像中的任意实例的能力。该定位引导器通过使用一款现成的物体检测器的监督和一种针对检测器未涵盖的（新颖）类别的自我训练方案，将类别名称的文本嵌入与扩散模型的区域视觉特征对齐进行训练。这一增强版的扩散模型被称为 InstaGen，可用作目标检测的数据合成器。我们进行了详尽的实验，结果显示在从 InstaGen 的合成数据集上进行训练时，目标检测器可以得到增强，在开放词汇（+4.5 AP）和数据稀缺（+1.2 至 5.2 AP）的情景中表现出比现有最先进方法更好的性能。