With the advent of universal function approximators in the domain of
reinforcement learning, the number of practical applications leveraging deep
reinforcement learning (DRL) has exploded. Decision-making in automated driving
tasks has emerged as a chief application among them, taking the sensor data or
the higher-order kinematic variables as the input and providing a discrete
choice or continuous control output. However, the black-box nature of the
models presents an overwhelming limitation that restricts the real-world
deployment of DRL in autonomous vehicles (AVs). Therefore, in this research
work, we focus on the interpretability of an attention-based DRL framework. We
use a continuous proximal policy optimization-based DRL algorithm as the
baseline model and add a multi-head attention framework in an open-source AV
simulation environment. We provide some analytical techniques for discussing
the interpretability of the trained models in terms of explainability and
causality for spatial and temporal correlations. We show that the weights in
the first head encode the positions of the neighboring vehicles while the
second head focuses on the leader vehicle exclusively. Also, the ego vehicle's
action is causally dependent on the vehicles in the target lane spatially and
temporally. Through these findings, we reliably show that these techniques can
help practitioners decipher the results of the DRL algorithms.

自适应驾驶决策中，深度强化学习模型的黑盒特性限制了实际应用，因此本研究工作关注一种基于注意力机制的可解释深度强化学习框架，并使用连续近端策略优化算法作为基准模型，在开源自适应驾驶仿真环境中加入了多头注意力机制。通过分析技术讨论模型的可解释性和因果关系，发现模型的第一个头部编码了邻近车辆的位置，而第二个头部则专注于领导车辆。此外，自车的动作在时空上受到目标车道上车辆的因果依赖。这些发现可帮助从业人员解读深度强化学习算法的结果。

深度强化学习自动驾驶决策的揭秘

Demystifying Deep Reinforcement Learning-Based Autonomous Vehicle  Decision-Making

We present a general class of machine learning algorithms called parametric
matrix models. Parametric matrix models are based on matrix equations, and the
design is motivated by the efficiency of reduced basis methods for
approximating solutions of parametric equations. The dependent variables can be
defined implicitly or explicitly, and the equations may use algebraic,
differential, or integral relations. Parametric matrix models can be trained
with empirical data only, and no high-fidelity model calculations are needed.
While originally designed for scientific computing, parametric matrix models
are universal function approximators that can be applied to general machine
learning problems. After introducing the underlying theory, we apply parametric
matrix models to a series of different challenges that show their performance
for a wide range of problems. For all the challenges tested here, parametric
matrix models produce accurate results within a computational framework that
allows for parameter extrapolation and interpretability.

我们提出了一种称为参数矩阵模型的通用机器学习算法类。参数矩阵模型基于矩阵方程，其设计受到了近似解参数方程的简化基准方法的高效性的启发。该模型的因变量可以隐式或显式定义，并且方程可以使用代数、微分或积分关系。参数矩阵模型只需要经验数据进行训练，不需要高保真模型计算。尽管最初设计用于科学计算，参数矩阵模型是通用的函数近似器，可用于解决一般的机器学习问题。在介绍了潜在理论后，我们将参数矩阵模型应用于一系列不同的挑战，展示了它们在广泛的问题上的性能。对于在这里测试的所有挑战，参数矩阵模型在允许参数外推和解释性的计算框架内产生准确的结果。

参数矩阵模型

Parametric Matrix Models

Neural networks have the ability to serve as universal function
approximators, but they are not interpretable and don't generalize well outside
of their training region. Both of these issues are problematic when trying to
apply standard neural ordinary differential equations (neural ODEs) to
dynamical systems. We introduce the polynomial neural ODE, which is a deep
polynomial neural network inside of the neural ODE framework. We demonstrate
the capability of polynomial neural ODEs to predict outside of the training
region, as well as perform direct symbolic regression without additional tools
such as SINDy.

本研究介绍了使用多项式神经常微分方程（polynomial neural ODE）框架解决神经常微分方程（neural ODE）应用中，解释性和泛化能力问题的方法，并且演示了多项式神经常微分方程在预测能力和符号回归方面的优秀性。