Deep reinforcement learning includes a broad family of algorithms that
parameterise an internal representation, such as a value function or policy, by
a deep neural network. Each algorithm optimises its parameters with respect to
an objective, such as Q-learning or policy gradient, that defines its
semantics. In this work, we propose an algorithm based on meta-gradient descent
that discovers its own objective, flexibly parameterised by a deep neural
network, solely from interactive experience with its environment. Over time,
this allows the agent to learn how to learn increasingly effectively.
Furthermore, because the objective is discovered online, it can adapt to
changes over time. We demonstrate that the algorithm discovers how to address
several important issues in RL, such as bootstrapping, non-stationarity, and
off-policy learning. On the Atari Learning Environment, the meta-gradient
algorithm adapts over time to learn with greater efficiency, eventually
outperforming the median score of a strong actor-critic baseline.

本文介绍的一种基于元梯度下降的算法可以通过与环境的交互经验发现自己的目标，并灵活地以深度神经网络为参数。随着时间的推移，该算法可以学习如何学习，最终在 Atari Learning Environment 上表现出比强化学习基准测试更高的得分。