Recent advances in high-fidelity simulators have enabled closed-loop training
of autonomous driving agents, potentially solving the distribution shift in
training v.s. deployment and allowing training to be scaled both safely and
cheaply. However, there is a lack of understanding of how to build effective
training benchmarks for closed-loop training. In this work, we present the
first empirical study which analyzes the effects of different training
benchmark designs on the success of learning agents, such as how to design
traffic scenarios and scale training environments. Furthermore, we show that
many popular RL algorithms cannot achieve satisfactory performance in the
context of autonomous driving, as they lack long-term planning and take an
extremely long time to train. To address these issues, we propose trajectory
value learning (TRAVL), an RL-based driving agent that performs planning with
multistep look-ahead and exploits cheaply generated imagined data for efficient
learning. Our experiments show that TRAVL can learn much faster and produce
safer maneuvers compared to all the baselines. For more information, visit the
project website: this https URL

本文针对自动驾驶代理的封闭式训练提出了一种新方法 —— 轨迹价值学习 (TRAVL)。采取多步预测进行规划，同时利用虚拟数据进行有效学习。实验证明，TRAVL 相比于其他模型，能够更快地学习并产生更安全的机动操作。