In this work, we propose the world's first closed-loop ML-based planning
benchmark for autonomous driving. While there is a growing body of ML-based
motion planners, the lack of established datasets and metrics has limited the
progress in this area. Existing benchmarks for autonomous vehicle motion
prediction have focused on short-term motion forecasting, rather than long-term
planning. This has led previous works to use open-loop evaluation with L2-based
metrics, which are not suitable for fairly evaluating long-term planning. Our
benchmark overcomes these limitations by introducing a large-scale driving
dataset, lightweight closed-loop simulator, and motion-planning-specific
metrics. We provide a high-quality dataset with 1500h of human driving data
from 4 cities across the US and Asia with widely varying traffic patterns
(Boston, Pittsburgh, Las Vegas and Singapore). We will provide a closed-loop
simulation framework with reactive agents and provide a large set of both
general and scenario-specific planning metrics. We plan to release the dataset
at NeurIPS 2021 and organize benchmark challenges starting in early 2022.

本文提出了第一个基于机器学习的自动驾驶闭环规划基准测试，介绍了一个数据集、模拟器和规划特定度量的框架。数据集包含来自美国和亚洲四个城市的 1500 小时的人类驾驶数据，模拟器使用反应性代理程序，提供了大量的通用和特定场景的规划度量。