Embodied scene understanding serves as the cornerstone for autonomous agents
to perceive, interpret, and respond to open driving scenarios. Such
understanding is typically founded upon Vision-Language Models (VLMs).
Nevertheless, existing VLMs are restricted to the 2D domain, devoid of spatial
awareness and long-horizon extrapolation proficiencies. We revisit the key
aspects of autonomous driving and formulate appropriate rubrics. Hereby, we
introduce the Embodied Language Model (ELM), a comprehensive framework tailored
for agents' understanding of driving scenes with large spatial and temporal
spans. ELM incorporates space-aware pre-training to endow the agent with robust
spatial localization capabilities. Besides, the model employs time-aware token
selection to accurately inquire about temporal cues. We instantiate ELM on the
reformulated multi-faced benchmark, and it surpasses previous state-of-the-art
approaches in all aspects. All code, data, and models will be publicly shared.

自主代理人通过具有大空间和时间跨度的驾驶场景的理解而享有实体化的场景理解，它包括具有鲁棒的空间定位能力和准确探询时间线索的嵌入式语言模型（ELM）。