Practising and honing skills forms a fundamental component of how humans learn, yet artificial agents are rarely specifically trained to perform them. Instead, they are usually trained end-to-end, with the hope being that useful skills will be implicitly learned in order to maximise discounted return of some extrinsic reward function. In this paper, we investigate how skills can be incorporated into the training of reinforcement learning (RL) agents in complex environments with large state-action spaces and sparse rewards. To this end, we created SkillHack, a benchmark of tasks and associated skills based on the game of NetHack. We evaluate a number of baselines on this benchmark, as well as our own novel skill-based method Hierarchical Kickstarting (HKS), which is shown to outperform all other evaluated methods. Our experiments show that learning with a prior knowledge of useful skills can significantly improve the performance of agents on complex problems. We ultimately argue that utilising predefined skills provides a useful inductive bias for RL problems, especially those with large state-action spaces and sparse rewards.

本文研究了如何将技能融入强化学习智能体的训练，提出了基于技能的新方法Hierarchical Kickstarting（HKS），并在复杂环境下的游戏NetHack及其他基线测试中验证了其表现优于其他方法，并认为利用预定义技能为具有大的状态-动作空间和稀疏奖励的强化学习问题提供了一个有用的归纳偏差。

层次化启动在强化学习技能转移中的应用