With the advent of universal function approximators in the domain of
reinforcement learning, the number of practical applications leveraging deep
reinforcement learning (DRL) has exploded. Decision-making in automated driving
tasks has emerged as a chief application among them, taking the sensor data or
the higher-order kinematic variables as the input and providing a discrete
choice or continuous control output. However, the black-box nature of the
models presents an overwhelming limitation that restricts the real-world
deployment of DRL in autonomous vehicles (AVs). Therefore, in this research
work, we focus on the interpretability of an attention-based DRL framework. We
use a continuous proximal policy optimization-based DRL algorithm as the
baseline model and add a multi-head attention framework in an open-source AV
simulation environment. We provide some analytical techniques for discussing
the interpretability of the trained models in terms of explainability and
causality for spatial and temporal correlations. We show that the weights in
the first head encode the positions of the neighboring vehicles while the
second head focuses on the leader vehicle exclusively. Also, the ego vehicle's
action is causally dependent on the vehicles in the target lane spatially and
temporally. Through these findings, we reliably show that these techniques can
help practitioners decipher the results of the DRL algorithms.

自适应驾驶决策中，深度强化学习模型的黑盒特性限制了实际应用，因此本研究工作关注一种基于注意力机制的可解释深度强化学习框架，并使用连续近端策略优化算法作为基准模型，在开源自适应驾驶仿真环境中加入了多头注意力机制。通过分析技术讨论模型的可解释性和因果关系，发现模型的第一个头部编码了邻近车辆的位置，而第二个头部则专注于领导车辆。此外，自车的动作在时空上受到目标车道上车辆的因果依赖。这些发现可帮助从业人员解读深度强化学习算法的结果。