Large Vision-Language Models (LVLMs), due to the remarkable visual reasoning
ability to understand images and videos, have received widespread attention in
the autonomous driving domain, which significantly advances the development of
interpretable end-to-end autonomous driving. However, current evaluations of
LVLMs primarily focus on the multi-faceted capabilities in common scenarios,
lacking quantifiable and automated assessment in autonomous driving contexts,
let alone severe road corner cases that even the state-of-the-art autonomous
driving perception systems struggle to handle. In this paper, we propose
CODA-LM, a novel vision-language benchmark for self-driving, which provides the
first automatic and quantitative evaluation of LVLMs for interpretable
autonomous driving including general perception, regional perception, and
driving suggestions. CODA-LM utilizes the texts to describe the road images,
exploiting powerful text-only large language models (LLMs) without image inputs
to assess the capabilities of LVLMs in autonomous driving scenarios, which
reveals stronger alignment with human preferences than LVLM judges. Experiments
demonstrate that even the closed-sourced commercial LVLMs like GPT-4V cannot
deal with road corner cases well, suggesting that we are still far from a
strong LVLM-powered intelligent driving agent, and we hope our CODA-LM can
become the catalyst to promote future development.

提出了 CODA-LM，这是一个用于自动驾驶的新兴视觉 - 语言基准测试，通过利用文本而不使用图像输入，使强大的文本 - 语言模型能够评估 LVLM 在自动驾驶场景中的能力，并揭示了它们与人类偏好的更强的一致性。实验证明，即使是像 GPT-4V 这样的商用 LVLM 也无法很好地处理路上的特殊情况，这表明我们离强大的 LVLM 驱动的智能驾驶代理系统还有很长的路要走，希望 CODA-LM 能够成为推动未来发展的催化剂。