Goals for reinforcement learning problems are typically defined through
hand-specified rewards. To design such problems, developers of learning
algorithms must inherently be aware of what the task goals are, yet we often
require agents to discover them on their own without any supervision beyond
these sparse rewards. While much of the power of reinforcement learning derives
from the concept that agents can learn with little guidance, this requirement
greatly burdens the training process. If we relax this one restriction and
endow the agent with knowledge of the reward function, and in particular of the
goal, we can leverage backwards induction to accelerate training. To achieve
this, we propose training a model to learn to take imagined reversal steps from
known goal states. Rather than training an agent exclusively to determine how
to reach a goal while moving forwards in time, our approach travels backwards
to jointly predict how we got there. We evaluate our work in Gridworld and
Towers of Hanoi and empirically demonstrate that it yields better performance
than standard DDQN.

通过训练一个模型来从已知的目标状态开始进行逆向预测，将强化学习中的目标函数引入到代理中，从而加速训练过程，并在 Gridworld 和汉诺塔游戏中进行了实验验证。