In a recent study, Reinforcement Learning (RL) used in combination with
many-objective search, has been shown to outperform alternative techniques
(random search and many-objective search) for online testing of Deep Neural
Network-enabled systems. The empirical evaluation of these techniques was
conducted on a state-of-the-art Autonomous Driving System (ADS). This work is a
replication and extension of that empirical study. Our replication shows that
RL does not outperform pure random test generation in a comparison conducted
under the same settings of the original study, but with no confounding factor
coming from the way collisions are measured. Our extension aims at eliminating
some of the possible reasons for the poor performance of RL observed in our
replication: (1) the presence of reward components providing contrasting or
useless feedback to the RL agent; (2) the usage of an RL algorithm (Q-learning)
which requires discretization of an intrinsically continuous state space.
Results show that our new RL agent is able to converge to an effective policy
that outperforms random testing. Results also highlight other possible
improvements, which open to further investigations on how to best leverage RL
for online ADS testing.

复制和扩展了前一次经验研究，研究表明引入强化学习与多目标搜索相结合的方式，在深度神经网络系统的在线测试中优于其他技术（随机搜索和多目标搜索）。新的 RL 代理能够收敛到一个有效策略，明显优于随机测试，并且揭示了进一步探索如何充分利用 RL 进行在线 ADS 测试的可能改进方向。

强化学习用于自动驾驶系统在线测试的复制与扩展研究

Reinforcement Learning for Online Testing of Autonomous Driving Systems:  a Replication and Extension Study

Recently, researches have explored the graph neural network (GNN) techniques
on text classification, since GNN does well in handling complex structures and
preserving global information. However, previous methods based on GNN are
mainly faced with the practical problems of fixed corpus level graph structure
which do not support online testing and high memory consumption. To tackle the
problems, we propose a new GNN based model that builds graphs for each input
text with global parameters sharing instead of a single graph for the whole
corpus. This method removes the burden of dependence between an individual text
and entire corpus which support online testing, but still preserve global
information. Besides, we build graphs by much smaller windows in the text,
which not only extract more local features but also significantly reduce the
edge numbers as well as memory consumption. Experiments show that our model
outperforms existing models on several text classification datasets even with
consuming less memory.

提出了一种新的基于图神经网络的模型，通过全局参数共享建立每个输入文本的图形而不是整个语料库的单个图形，这种方法支持在线测试，同时提取更多的本地特征和显著降低边缘数量以及内存消耗，在多个文本分类数据集上性能优于现有模型。