Referring expression comprehension (REC) involves localizing a target
instance based on a textual description. Recent advancements in REC have been
driven by large multimodal models (LMMs) like CogVLM, which achieved 92.44%
accuracy on RefCOCO. However, this study questions whether existing benchmarks
such as RefCOCO, RefCOCO+, and RefCOCOg, capture LMMs' comprehensive
capabilities. We begin with a manual examination of these benchmarks, revealing
high labeling error rates: 14% in RefCOCO, 24% in RefCOCO+, and 5% in RefCOCOg,
which undermines the authenticity of evaluations. We address this by excluding
problematic instances and reevaluating several LMMs capable of handling the REC
task, showing significant accuracy improvements, thus highlighting the impact
of benchmark noise. In response, we introduce Ref-L4, a comprehensive REC
benchmark, specifically designed to evaluate modern REC models. Ref-L4 is
distinguished by four key features: 1) a substantial sample size with 45,341
annotations; 2) a diverse range of object categories with 365 distinct types
and varying instance scales from 30 to 3,767; 3) lengthy referring expressions
averaging 24.2 words; and 4) an extensive vocabulary comprising 22,813 unique
words. We evaluate a total of 24 large models on Ref-L4 and provide valuable
insights. The cleaned versions of RefCOCO, RefCOCO+, and RefCOCOg, as well as
our Ref-L4 benchmark and evaluation code, are available at
this https URL

参考表达理解（REC）涉及根据文本描述定位目标实例。本研究质疑现有基准对大型多模态模型（LMMs）的全面性能力，通过手动检查现有基准，发现高标注错误率，并通过排除问题实例和重新评估多种 LMMs 来解决。为了评估现代 REC 模型，引入了 Ref-L4，一个全面的 REC 基准。在 Ref-L4 上评估了 24 个大模型并提供了有价值的见解。