We develop a deep reinforcement learning framework for tactical decision
making in an autonomous truck, specifically for Adaptive Cruise Control (ACC)
and lane change maneuvers in a highway scenario. Our results demonstrate that
it is beneficial to separate high-level decision-making processes and low-level
control actions between the reinforcement learning agent and the low-level
controllers based on physical models. In the following, we study optimizing the
performance with a realistic and multi-objective reward function based on Total
Cost of Operation (TCOP) of the truck using different approaches; by adding
weights to reward components, by normalizing the reward components and by using
curriculum learning techniques.

我们发展了一个深度强化学习框架，专门用于自动卡车的战术决策，尤其是自适应巡航控制和高速公路上的车道变换动作。

基于全面运营成本回报的深度强化学习自主卡车战术决策

Tactical Decision Making for Autonomous Trucks by Deep Reinforcement  Learning with Total Cost of Operation Based Reward

Tactical decision making is a critical feature for advanced driving systems,
that incorporates several challenges such as complexity of the uncertain
environment and reliability of the autonomous system. In this work, we develop
a multi-modal architecture that includes the environmental modeling of ego
surrounding and train a deep reinforcement learning (DRL) agent that yields
consistent performance in stochastic highway driving scenarios. To this end, we
feed the occupancy grid of the ego surrounding into the DRL agent and obtain
the high-level sequential commands (i.e. lane change) to send them to
lower-level controllers. We will show that dividing the autonomous driving
problem into a multi-layer control architecture enables us to leverage the AI
power to solve each layer separately and achieve an admissible reliability
score. Comparing with end-to-end approaches, this architecture enables us to
end up with a more reliable system which can be implemented in actual
self-driving cars.

本研究开发了一个多模态架构，包括自我周围环境的建模，并训练了一个深度强化学习代理，以在随机高速公路驾驶场景中实现一致的性能，将自主驾驶问题分成多层控制结构使我们能够将 AI 的能力分别用于解决每一层，最终实现更可靠的系统，可在实际的自动驾驶汽车中实现。

使用深度强化学习的自主驾驶顺序决策的分层体系结构

A Hierarchical Architecture for Sequential Decision-Making in Autonomous  Driving using Deep Reinforcement Learning

Tactical decision making for autonomous driving is challenging due to the
diversity of environments, the uncertainty in the sensor information, and the
complex interaction with other road users. This paper introduces a general
framework for tactical decision making, which combines the concepts of planning
and learning, in the form of Monte Carlo tree search and deep reinforcement
learning. The method is based on the AlphaGo Zero algorithm, which is extended
to a domain with a continuous state space where self-play cannot be used. The
framework is applied to two different highway driving cases in a simulated
environment and it is shown to perform better than a commonly used baseline
method. The strength of combining planning and learning is also illustrated by
a comparison to using the Monte Carlo tree search or the neural network policy
separately.

该论文介绍了一种将规划和学习结合起来的决策框架，通过蒙特卡罗树搜索和深度强化学习，并将其应用于高速公路驾驶案例，并与基线方法进行比较。结果表明该框架综合了规划和学习的优势，能够更好地应对自动驾驶中具有挑战性的决策制定问题。