Self-evolution is indispensable to realize full autonomous driving. This
paper presents a self-evolving decision-making system based on the Integrated
Decision and Control (IDC), an advanced framework built on reinforcement
learning (RL). First, an RL algorithm called constrained mixed policy gradient
(CMPG) is proposed to consistently upgrade the driving policy of the IDC. It
adapts the MPG under the penalty method so that it can solve constrained
optimization problems using both the data and model. Second, an attention-based
encoding (ABE) method is designed to tackle the state representation issue. It
introduces an embedding network for feature extraction and a weighting network
for feature fusion, fulfilling order-insensitive encoding and importance
distinguishing of road users. Finally, by fusing CMPG and ABE, we develop the
first data-driven decision and control system under the IDC architecture, and
deploy the system on a fully-functional self-driving vehicle running in daily
operation. Experiment results show that boosting by data, the system can
achieve better driving ability over model-based methods. It also demonstrates
safe, efficient and smart driving behavior in various complex scenes at a
signalized intersection with real mixed traffic flow.

本文提出了一种基于 RL 算法的自我进化决策系统，包含 CMPG 和 ABE 技术，该系统被部署在一辆自动驾驶汽车上，实现了智能、高效、安全的驾驶行为。实验表明，该系统可显著提升驾驶能力和模型方法相比具有更好的驾驶环境适应性。