We study building a multi-task agent in Minecraft. Without human
demonstrations, solving long-horizon tasks in this open-ended environment with
reinforcement learning (RL) is extremely sample inefficient. To tackle the
challenge, we decompose solving Minecraft tasks into learning basic skills and
planning over the skills. We propose three types of fine-grained basic skills
in Minecraft, and use RL with intrinsic rewards to accomplish basic skills with
high success rates. For skill planning, we use Large Language Models to find
the relationships between skills and build a skill graph in advance. When the
agent is solving a task, our skill search algorithm walks on the skill graph
and generates the proper skill plans for the agent. In experiments, our method
accomplishes 24 diverse Minecraft tasks, where many tasks require sequentially
executing for more than 10 skills. Our method outperforms baselines in most
tasks by a large margin. The project's website and code can be found at
this https URL.

在 Minecraft 环境中，我们使用强化学习技术训练多任务代理，将任务分解成学习基本技能和规划技能这两个步骤，通过提出三种细粒度的基本技能，搭配内在奖励机制使用强化学习训练代理，在学会技能后，我们使用大型语言模型建立技能图模型，并提出技能搜索算法为代理生成适当的技能计划。在实验中，我们成功的完成了 24 个不同的 Minecraft 任务，并得到了比基线算法更好的表现。

Plan4MC：针对开放世界 Minecraft 任务的技能强化学习和规划

Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks

The objective of lifelong reinforcement learning (RL) is to optimize agents
which can continuously adapt and interact in changing environments. However,
current RL approaches fail drastically when environments are non-stationary and
interactions are non-episodic. We propose Lifelong Skill Planning (LiSP), an
algorithmic framework for non-episodic lifelong RL based on planning in an
abstract space of higher-order skills. We learn the skills in an unsupervised
manner using intrinsic rewards and plan over the learned skills using a learned
dynamics model. Moreover, our framework permits skill discovery even from
offline data, thereby reducing the need for excessive real-world interactions.
We demonstrate empirically that LiSP successfully enables long-horizon planning
and learns agents that can avoid catastrophic failures even in challenging
non-stationary and non-episodic environments derived from gridworld and MuJoCo
benchmarks.

论文提出了一种基于高阶技能抽象规划的无剧本生涯强化学习框架 LiSP，该框架使用内在奖励进行技能学习并使用学到的动力学模型进行规划，能够在不断变化的非稳态非剧本的环境中取得成功，甚至有助于从离线数据中发现技能，减少与现实世界的交互。