The objective of lifelong reinforcement learning (RL) is to optimize agents
which can continuously adapt and interact in changing environments. However,
current RL approaches fail drastically when environments are non-stationary and
interactions are non-episodic. We propose Lifelong Skill Planning (LiSP), an
algorithmic framework for non-episodic lifelong RL based on planning in an
abstract space of higher-order skills. We learn the skills in an unsupervised
manner using intrinsic rewards and plan over the learned skills using a learned
dynamics model. Moreover, our framework permits skill discovery even from
offline data, thereby reducing the need for excessive real-world interactions.
We demonstrate empirically that LiSP successfully enables long-horizon planning
and learns agents that can avoid catastrophic failures even in challenging
non-stationary and non-episodic environments derived from gridworld and MuJoCo
benchmarks.

论文提出了一种基于高阶技能抽象规划的无剧本生涯强化学习框架 LiSP，该框架使用内在奖励进行技能学习并使用学到的动力学模型进行规划，能够在不断变化的非稳态非剧本的环境中取得成功，甚至有助于从离线数据中发现技能，减少与现实世界的交互。