Learning tabula rasa, that is without any prior knowledge, is the prevalent
workflow in reinforcement learning (RL) research. However, RL systems, when
applied to large-scale settings, rarely operate tabula rasa. Such large-scale
systems undergo multiple design or algorithmic changes during their development
cycle and use ad hoc approaches for incorporating these changes without
re-training from scratch, which would have been prohibitively expensive.
Additionally, the inefficiency of deep RL typically excludes researchers
without access to industrial-scale resources from tackling
computationally-demanding problems. To address these issues, we present
reincarnating RL as an alternative workflow or class of problem settings, where
prior computational work (e.g., learned policies) is reused or transferred
between design iterations of an RL agent, or from one RL agent to another. As a
step towards enabling reincarnating RL from any agent to any other agent, we
focus on the specific setting of efficiently transferring an existing
sub-optimal policy to a standalone value-based RL agent. We find that existing
approaches fail in this setting and propose a simple algorithm to address their
limitations. Equipped with this algorithm, we demonstrate reincarnating RL's
gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task,
and the real-world problem of navigating stratospheric balloons. Overall, this
work argues for an alternative approach to RL research, which we believe could
significantly improve real-world RL adoption and help democratize it further.
Open-sourced code and trained agents at
this https URL.

提出转世强化学习理论作为强化学习（RL）研究的另一种工作流程或问题类别，以便能够在设计迭代之间或不同代理之间重复使用或转移先前的计算工作（例如，学习策略）; 研究关注现有子优政府的有效传输，并提出了一种简单的算法来解决其限制，最后，在 Atari 2600 游戏，挑战性的互动任务以及导航平流层气球等问题上展现了转世 RL 与 tabula rasa RL 相比的优势，证明这是一种可能提高 RL 实际应用和进一步民主化 RL 的另一种方法。