Developing and testing automated driving models in the real world might be
challenging and even dangerous, while simulation can help with this, especially
for challenging maneuvers. Deep reinforcement learning (DRL) has the potential
to tackle complex decision-making and controlling tasks through learning and
interacting with the environment, thus it is suitable for developing automated
driving while not being explored in detail yet. This study carried out a
comprehensive study by implementing, evaluating, and comparing the two DRL
algorithms, Deep Q-networks (DQN) and Trust Region Policy Optimization (TRPO),
for training automated driving on the highway-env simulation platform.
Effective and customized reward functions were developed and the implemented
algorithms were evaluated in terms of onlane accuracy (how well the car drives
on the road within the lane), efficiency (how fast the car drives), safety (how
likely the car is to crash into obstacles), and comfort (how much the car makes
jerks, e.g., suddenly accelerates or brakes). Results show that the TRPO-based
models with modified reward functions delivered the best performance in most
cases. Furthermore, to train a uniform driving model that can tackle various
driving maneuvers besides the specific ones, this study expanded the
highway-env and developed an extra customized training environment, namely,
ComplexRoads, integrating various driving maneuvers and multiple road scenarios
together. Models trained on the designed ComplexRoads environment can adapt
well to other driving maneuvers with promising overall performance. Lastly,
several functionalities were added to the highway-env to implement this work.
The codes are open on GitHub at this https URL

本研究通过实现、评估和比较两种深度强化学习算法（Deep Q-networks 和 Trust Region Policy Optimization）来训练自动驾驶车辆，以及开发和应用奖励函数，并在基于模拟环境的高速公路自动驾驶训练平台中进行评估，结果显示 TRPO 算法在大多数情况下以及将多种驾驶演练和多种路况情况集成于一体的 ComplexRoads 训练环境中效果最佳。