In recent research, significant attention has been devoted to the
open-vocabulary object detection task, aiming to generalize beyond the limited
number of classes labeled during training and detect objects described by
arbitrary category names at inference. Compared with conventional object
detection, open vocabulary object detection largely extends the object
detection categories. However, it relies on calculating the similarity between
image regions and a set of arbitrary category names with a pretrained
vision-and-language model. This implies that, despite its open-set nature, the
task still needs the predefined object categories during the inference stage.
This raises the question: What if we do not have exact knowledge of object
categories during inference? In this paper, we call such a new setting as
generative open-ended object detection, which is a more general and practical
problem. To address it, we formulate object detection as a generative problem
and propose a simple framework named GenerateU, which can detect dense objects
and generate their names in a free-form way. Particularly, we employ Deformable
DETR as a region proposal generator with a language model translating visual
regions to object names. To assess the free-form object detection task, we
introduce an evaluation method designed to quantitatively measure the
performance of generative outcomes. Extensive experiments demonstrate strong
zero-shot detection performance of our GenerateU. For example, on the LVIS
dataset, our GenerateU achieves comparable results to the open-vocabulary
object detection method GLIP, even though the category names are not seen by
GenerateU during inference. Code is available at: https://
github.com/FoundationVision/GenerateU .

生成式开放式物体检测是一种更通用、实用的问题，本论文提出了一个名为 GenerateU 的简单框架，将物体检测作为一个生成问题，可以以自由形式检测密集物体并生成它们的名称，通过广泛的实验验证了 GenerateU 的强大的零样本检测性能。