Meta-learning algorithms use past experience to learn to quickly solve new
tasks. In the context of reinforcement learning, meta-learning algorithms
acquire reinforcement learning procedures to solve new problems more
efficiently by utilizing experience from prior tasks. The performance of
meta-learning algorithms depends on the tasks available for meta-training: in
the same way that supervised learning generalizes best to test points drawn
from the same distribution as the training points, meta-learning methods
generalize best to tasks from the same distribution as the meta-training tasks.
In effect, meta-reinforcement learning offloads the design burden from
algorithm design to task design. If we can automate the process of task design
as well, we can devise a meta-learning algorithm that is truly automated. In
this work, we take a step in this direction, proposing a family of unsupervised
meta-learning algorithms for reinforcement learning. We motivate and describe a
general recipe for unsupervised meta-reinforcement learning, and present an
instantiation of this approach. Our conceptual and theoretical contributions
consist of formulating the unsupervised meta-reinforcement learning problem and
describing how task proposals based on mutual information can be used to train
optimal meta-learners. Our experimental results indicate that unsupervised
meta-reinforcement learning effectively acquires accelerated reinforcement
learning procedures without the need for manual task design and these
procedures exceed the performance of learning from scratch.

本研究提出了一系列用于强化学习的无监督元学习算法，以解决通过任务提案进行任务训练而非手动任务设计所带来的算法设计负担，并得出这些程序具有优异性能的结论。