We study building a multi-task agent in Minecraft. Without human demonstrations, solving long-horizon tasks in this open-ended environment with reinforcement learning (RL) is extremely sample inefficient. To tackle the challenge, we decompose solving Minecraft tasks into learning basic skills and planning over the skills. We propose three types of fine-grained basic skills in Minecraft, and use RL with intrinsic rewards to accomplish basic skills with high success rates. For skill planning, we use Large Language Models to find the relationships between skills and build a skill graph in advance. When the agent is solving a task, our skill search algorithm walks on the skill graph and generates the proper skill plans for the agent. In experiments, our method accomplishes 24 diverse Minecraft tasks, where many tasks require sequentially executing for more than 10 skills. Our method outperforms baselines in most tasks by a large margin. The project's website and code can be found at https://sites.google.com/view/plan4mc.

在Minecraft环境中，我们使用强化学习技术训练多任务代理，将任务分解成学习基本技能和规划技能这两个步骤，通过提出三种细粒度的基本技能，搭配内在奖励机制使用强化学习训练代理，在学会技能后，我们使用大型语言模型建立技能图模型，并提出技能搜索算法为代理生成适当的技能计划。在实验中，我们成功的完成了24个不同的Minecraft任务，并得到了比基线算法更好的表现。

Plan4MC：针对开放世界 Minecraft 任务的技能强化学习和规划