Humans commonly solve complex problems by decomposing them into easier
subproblems and then combining the subproblem solutions. This type of
compositional reasoning permits reuse of the subproblem solutions when tackling
future tasks that share part of the underlying compositional structure. In a
continual or lifelong reinforcement learning (RL) setting, this ability to
decompose knowledge into reusable components would enable agents to quickly
learn new RL tasks by leveraging accumulated compositional structures. We
explore a particular form of composition based on neural modules and present a
set of RL problems that intuitively admit compositional solutions. Empirically,
we demonstrate that neural composition indeed captures the underlying structure
of this space of problems. We further propose a compositional lifelong RL
method that leverages accumulated neural components to accelerate the learning
of future tasks while retaining performance on previous tasks via off-line RL
over replayed experiences.

本文提出了一种基于神经模块的组成式学习方式，探讨了在连续强化学习模型下的可重复利用子问题解决方案，并且通过离线重放经验保持表现以加速未来任务学习的累积神经组件复合持续强化学习方法。