Meta reinforcement learning (RL) attempts to discover new RL algorithms
automatically from environment interaction. In so-called black-box approaches,
the policy and the learning algorithm are jointly represented by a single
neural network. These methods are very flexible, but they tend to underperform
in terms of generalisation to new, unseen environments. In this paper, we
explore the role of symmetries in meta-generalisation. We show that a recent
successful meta RL approach that meta-learns an objective for
backpropagation-based learning exhibits certain symmetries (specifically the
reuse of the learning rule, and invariance to input and output permutations)
that are not present in typical black-box meta RL systems. We hypothesise that
these symmetries can play an important role in meta-generalisation. Building
off recent work in black-box supervised meta learning, we develop a black-box
meta RL system that exhibits these same symmetries. We show through careful
experimentation that incorporating these symmetries can lead to algorithms with
a greater ability to generalise to unseen action & observation spaces, tasks,
and environments.

本文研究了元强化学习中的对称性在元泛化中的作用，我们发现对称性和黑盒元强化学习系统中通常不存在的神经网络可以帮助提高算法泛化性能。通过实验验证，加入这些对称性可以使算法更好地推广到未见过的行动和观测空间、任务和环境之中。