Autonomous driving is a complex and challenging task that aims at safe motion
planning through scene understanding and reasoning. While vision-only
autonomous driving methods have recently achieved notable performance, through
enhanced scene understanding, several key issues, including lack of reasoning,
low generalization performance and long-tail scenarios, still need to be
addressed. In this paper, we present VLP, a novel Vision-Language-Planning
framework that exploits language models to bridge the gap between linguistic
understanding and autonomous driving. VLP enhances autonomous driving systems
by strengthening both the source memory foundation and the self-driving car's
contextual understanding. VLP achieves state-of-the-art end-to-end planning
performance on the challenging NuScenes dataset by achieving 35.9\% and 60.5\%
reduction in terms of average L2 error and collision rates, respectively,
compared to the previous best method. Moreover, VLP shows improved performance
in challenging long-tail scenarios and strong generalization capabilities when
faced with new urban environments.

自主驾驶中的场景理解和推理是复杂而具有挑战性的任务。本文提出了一种新颖的 Vision-Language-Planning（VLP）框架，通过利用自然语言模型来弥合语义理解和自主驾驶之间的鸿沟，从而增强自动驾驶系统的规划性能，提高在挑战性场景中的表现，以及在面对新型城市环境时的强大泛化能力。