The challenge of Out-Of-Distribution (OOD) robustness remains a critical
hurdle towards deploying deep vision models. Open-vocabulary object detection
extends the capabilities of traditional object detection frameworks to
recognize and classify objects beyond predefined categories. Investigating OOD
robustness in open-vocabulary object detection is essential to increase the
trustworthiness of these models. This study presents a comprehensive robustness
comparison of zero-shot capabilities of three recent open-vocabulary foundation
object detection models, namely OWL-ViT, YOLO World, and Grounding DINO.
Experiments carried out on the COCO-O and COCO-C benchmarks encompassing
distribution shifts highlight the challenges of the models' robustness. Source
code shall be made available to the research community on GitHub.

该研究提供了对三种最新的开放词汇基础目标检测模型（OWL-ViT、YOLO World 和 Grounding DINO）的零样本能力的全面鲁棒性比较，通过在包含分布转移的 COCO-O 和 COCO-C 基准上进行实验揭示了模型鲁棒性的挑战。

分布变化下基于开放词汇的基础对象检测鲁棒性研究

Investigating Robustness of Open-Vocabulary Foundation Object Detectors  under Distribution Shifts

We introduce Grounded SAM, which uses Grounding DINO as an open-set object
detector to combine with the segment anything model (SAM). This integration
enables the detection and segmentation of any regions based on arbitrary text
inputs and opens a door to connecting various vision models. As shown in Fig.1,
a wide range of vision tasks can be achieved by using the versatile Grounded
SAM pipeline. For example, an automatic annotation pipeline based solely on
input images can be realized by incorporating models such as BLIP and Recognize
Anything. Additionally, incorporating Stable-Diffusion allows for controllable
image editing, while the integration of OSX facilitates promptable 3D human
motion analysis. Grounded SAM also shows superior performance on
open-vocabulary benchmarks, achieving 48.7 mean AP on SegInW (Segmentation in
the wild) zero-shot benchmark with the combination of Grounding DINO-Base and
SAM-Huge models.

引入地面 SAM，使用地面化 DINO 作为开放目标检测器并结合分段任何模型（SAM），实现基于任意文本输入的任何区域的检测和分割，并打开了连接各种视觉模型的大门。