In a recent study, Reinforcement Learning (RL) used in combination with many-objective search, has been shown to outperform alternative techniques (random search and many-objective search) for online testing of Deep Neural Network-enabled systems. The empirical evaluation of these techniques was conducted on a state-of-the-art Autonomous Driving System (ADS). This work is a replication and extension of that empirical study. Our replication shows that RL does not outperform pure random test generation in a comparison conducted under the same settings of the original study, but with no confounding factor coming from the way collisions are measured. Our extension aims at eliminating some of the possible reasons for the poor performance of RL observed in our replication: (1) the presence of reward components providing contrasting or useless feedback to the RL agent; (2) the usage of an RL algorithm (Q-learning) which requires discretization of an intrinsically continuous state space. Results show that our new RL agent is able to converge to an effective policy that outperforms random testing. Results also highlight other possible improvements, which open to further investigations on how to best leverage RL for online ADS testing.

复制和扩展了前一次经验研究，研究表明引入强化学习与多目标搜索相结合的方式，在深度神经网络系统的在线测试中优于其他技术（随机搜索和多目标搜索）。新的 RL 代理能够收敛到一个有效策略，明显优于随机测试，并且揭示了进一步探索如何充分利用 RL 进行在线 ADS 测试的可能改进方向。

强化学习用于自动驾驶系统在线测试的复制与扩展研究