Artificial neural networks used for reinforcement learning are structurally rigid, meaning that each optimized parameter of the network is tied to its specific placement in the network structure. It also means that a network only works with pre-defined and fixed input- and output sizes. This is a consequence of having the number of optimized parameters being directly dependent on the structure of the network. Structural rigidity limits the ability to optimize parameters of policies across multiple environments that do not share input and output spaces. Here, we evolve a set of neurons and plastic synapses each represented by a gated recurrent unit (GRU). During optimization, the parameters of these fundamental units of a neural network are optimized in different random structural configurations. Earlier work has shown that parameter sharing between units is important for making structurally flexible neurons We show that it is possible to optimize a set of distinct neuron- and synapse types allowing for a mitigation of the symmetry dilemma. We demonstrate this by optimizing a single set of neurons and synapses to solve multiple reinforcement learning control tasks simultaneously.

人工神经网络用于强化学习是结构僵化的，参数只能固定在预定义的网络结构中，这限制了在不同环境中优化参数的能力。在本研究中，我们通过演化一组神经元和可塑突触来解决这个问题，展示了通过优化不同的随机结构配置下的神经元的参数，可以解决多个强化学习控制任务。

结构灵活的神经网络：为普适智能体进化构建模块