Recent advancements in large vision-language models enabled visual object
detection in open-vocabulary scenarios, where object classes are defined in
free-text formats during inference. In this paper, we aim to probe the
state-of-the-art methods for open-vocabulary object detection to determine to
what extent they understand fine-grained properties of objects and their parts.
To this end, we introduce an evaluation protocol based on dynamic vocabulary
generation to test whether models detect, discern, and assign the correct
fine-grained description to objects in the presence of hard-negative classes.
We contribute with a benchmark suite of increasing difficulty and probing
different properties like color, pattern, and material. We further enhance our
investigation by evaluating several state-of-the-art open-vocabulary object
detectors using the proposed protocol and find that most existing solutions,
which shine in standard open-vocabulary benchmarks, struggle to accurately
capture and distinguish finer object details. We conclude the paper by
highlighting the limitations of current methodologies and exploring promising
research directions to overcome the discovered drawbacks. Data and code are
available at this https URL

我们在这篇论文中通过引入动态词汇生成的评估方案来探索现有开放词汇物体检测方法对物体的细粒度属性及其部分了解的程度，并通过对几种最先进的开放词汇物体检测器的评估，发现大多数现有方法难以准确捕捉和区分物体的细节，并总结了当前方法的局限性和有望克服这些缺点的研究方向。

细节决定成败：评估针对细节理解的开放词汇物体检测器

The devil is in the fine-grained details: Evaluating open-vocabulary  object detectors for fine-grained understanding

Looking at the world from above, it is possible to estimate many properties
of a given location, including the type of land cover and the expected land
use. Historically, such tasks have relied on relatively coarse-grained
categories due to the difficulty of obtaining fine-grained annotations. In this
work, we propose an easily extensible approach that makes it possible to
estimate fine-grained properties from overhead imagery. In particular, we
propose a cross-modal distillation strategy to learn to predict the
distribution of fine-grained properties from overhead imagery, without
requiring any manual annotation of overhead imagery. We show that our learned
models can be used directly for applications in mapping and image localization.

通过交叉模态蒸馏策略，从俯瞰图像中学习预测细粒度地物属性的分布，以实现精细化的土地覆盖估计，结果可应用于地图绘制和图像定位。