In this paper, we study the Tiered Reinforcement Learning setting, a parallel
transfer learning framework, where the goal is to transfer knowledge from the
low-tier (source) task to the high-tier (target) task to reduce the exploration
risk of the latter while solving the two tasks in parallel. Unlike previous
work, we do not assume the low-tier and high-tier tasks share the same dynamics
or reward functions, and focus on robust knowledge transfer without prior
knowledge on the task similarity. We identify a natural and necessary condition
called the "Optimal Value Dominance" for our objective. Under this condition,
we propose novel online learning algorithms such that, for the high-tier task,
it can achieve constant regret on partial states depending on the task
similarity and retain near-optimal regret when the two tasks are dissimilar,
while for the low-tier task, it can keep near-optimal without making sacrifice.
Moreover, we further study the setting with multiple low-tier tasks, and
propose a novel transfer source selection mechanism, which can ensemble the
information from all low-tier tasks and allow provable benefits on a much
larger state-action space.

本文研究分层强化学习的并行传输学习框架，提出了新的在线学习算法以及转移来源选择机制来实现对高层任务的常数后悔性，在多低层任务的情况下也能获得更大的状态行为空间的利益。