End-to-end autonomous driving recently emerged as a promising research
direction to target autonomy from a full-stack perspective. Along this line,
many of the latest works follow an open-loop evaluation setting on nuScenes to
study the planning behavior. In this paper, we delve deeper into the problem by
conducting thorough analyses and demystifying more devils in the details. We
initially observed that the nuScenes dataset, characterized by relatively
simple driving scenarios, leads to an under-utilization of perception
information in end-to-end models incorporating ego status, such as the ego
vehicle's velocity. These models tend to rely predominantly on the ego
vehicle's status for future path planning. Beyond the limitations of the
dataset, we also note that current metrics do not comprehensively assess the
planning quality, leading to potentially biased conclusions drawn from existing
benchmarks. To address this issue, we introduce a new metric to evaluate
whether the predicted trajectories adhere to the road. We further propose a
simple baseline able to achieve competitive results without relying on
perception annotations. Given the current limitations on the benchmark and
metrics, we suggest the community reassess relevant prevailing research and be
cautious whether the continued pursuit of state-of-the-art would yield
convincing and universal conclusions. Code and models are available at
https://github.com/NVlabs/BEV-Planner

自动驾驶最新研究中，终端到终端的全栈自动化日益成为一个有应用前景的研究方向。本文通过深入分析与详细阐述，探究了该问题的更多细节。研究表明，nuScenes 数据集在相对简单的驾驶场景中存在感知信息的低利用率，尤其是包括自身状态在内的整车速度对终端到终端模型的未来路径规划起到主导作用。此外，当前的评估指标无法全面评估规划质量，导致现有基准测试可能产生偏见。为了解决这个问题，我们引入了一种新的度量指标来评估预测轨迹是否符合道路要求。我们进一步提出了一种简单的基准模型，能够在不依赖感知注释的情况下取得有竞争力的结果。考虑到当前基准测试和指标的限制，我们建议学术界重新评估相关研究，并谨慎是否持续追求最新技术将带来令人信服和普适性较强的结论。