We propose and address a novel few-shot RL problem, where a task is
characterized by a subtask graph which describes a set of subtasks and their
dependencies that are unknown to the agent. The agent needs to quickly adapt to
the task over few episodes during adaptation phase to maximize the return in
the test phase. Instead of directly learning a meta-policy, we develop a
Meta-learner with Subtask Graph Inference(MSGI), which infers the latent
parameter of the task by interacting with the environment and maximizes the
return given the latent parameter. To facilitate learning, we adopt an
intrinsic reward inspired by upper confidence bound (UCB) that encourages
efficient exploration. Our experiment results on two grid-world domains and
StarCraft II environments show that the proposed method is able to accurately
infer the latent task parameter, and to adapt more efficiently than existing
meta RL and hierarchical RL methods.

本文提出了一种新颖的少样本强化学习问题，其通过任务的子任务图描述一组子任务及其依赖关系，我们开发了一种元学习器，即消息传递子任务图推理器（MSGI），该学习器通过与环境交互推断任务的潜在参数，并采用上限置信界中得到启发的内在奖励以促进有效探索，实验结果表明该方法能够准确推断潜在任务参数， 并比现有的元强化学习和分层强化学习方法更有效地适应。