Deep reinforcement learning (DRL) has emerged as a promising approach for
developing more intelligent autonomous vehicles (AVs). A typical DRL
application on AVs is to train a neural network-based driving policy. However,
the black-box nature of neural networks can result in unpredictable decision
failures, making such AVs unreliable. To this end, this work proposes a method
to identify and protect unreliable decisions of a DRL driving policy. The basic
idea is to estimate and constrain the policy's performance uncertainty, which
quantifies potential performance drop due to insufficient training data or
network fitting errors. By constraining the uncertainty, the DRL model's
performance is always greater than that of a baseline policy. The uncertainty
caused by insufficient data is estimated by the bootstrapped method. Then, the
uncertainty caused by the network fitting error is estimated using an ensemble
network. Finally, a baseline policy is added as the performance lower bound to
avoid potential decision failures. The overall framework is called
uncertainty-bound reinforcement learning (UBRL). The proposed UBRL is evaluated
on DRL policies with different amounts of training data, taking an unprotected
left-turn driving case as an example. The result shows that the UBRL method can
identify potentially unreliable decisions of DRL policy. The UBRL guarantees to
outperform baseline policy even when the DRL policy is not well-trained and has
high uncertainty. Meanwhile, the performance of UBRL improves with more
training data. Such a method is valuable for the DRL application on real-road
driving and provides a metric to evaluate a DRL policy.

该研究提出了一种基于不确定性边界的深度强化学习方法，用于保护无人驾驶决策的可靠性。该方法通过限制无人车神经网络决策策略的性能不确定性，从而提高了性能表现，并能够在不同程度的训练数据下有效地保护可靠性，并提高性能。