Language-based object detection is a promising direction towards building a natural interface to describe objects in images that goes far beyond plain category names. While recent methods show great progress in that direction, proper evaluation is lacking. With OmniLabel, we propose a novel task definition, dataset, and evaluation metric. The task subsumes standard- and open-vocabulary detection as well as referring expressions. With more than 28K unique object descriptions on over 25K images, OmniLabel provides a challenging benchmark with diverse and complex object descriptions in a naturally open-vocabulary setting. Moreover, a key differentiation to existing benchmarks is that our object descriptions can refer to one, multiple or even no object, hence, providing negative examples in free-form text. The proposed evaluation handles the large label space and judges performance via a modified average precision metric, which we validate by evaluating strong language-based baselines. OmniLabel indeed provides a challenging test bed for future research on language-based detection.

提出了新的任务定义、数据集和评估指标，包括标准和开放词汇检测以及引用表达式。OmniLabel 是具有挑战性的基准，具有多样化的复杂对象描述，并且是一个自然的开放词汇环境下的难点。评估使用经过修改的平均精度度量，通过评估强大的基线来验证.

OmniLabel: 一项挑战性的基于语言的物体检测基准测试