A key challenge for reinforcement learning (RL) consists of learning in environments with sparse extrinsic rewards. In contrast to current RL methods, humans are able to learn new skills with little or no reward by using various forms of intrinsic motivation. We propose AMIGo, a novel agent incorporating a goal-generating teacher that proposes Adversarially Motivated Intrinsic Goals to train a goal-conditioned "student" policy in the absence of (or alongside) environment reward. Specifically, through a simple but effective "constructively adversarial" objective, the teacher learns to propose increasingly challenging---yet achievable---goals that allow the student to learn general skills for acting in a new environment, independent of the task to be solved. We show that our method generates a natural curriculum of self-proposed goals which ultimately allows the agent to solve challenging procedurally-generated tasks where other forms of intrinsic motivation and state-of-the-art RL methods fail.

提出了一种基于元学习的AMIGo智能体，该智能体包含一个目标生成教师，该教师提出具有对抗性的内在目标来训练一个学生策略，使其在缺少环境奖励时可以学到普遍技能，通过自我建议的目标产生自然课程，并在固有动机和最新RL方法失败的情况下解决具有挑战性的生成式任务。

AMIGo学习: 对抗动机本质目标