As a marriage between offline RL and meta-RL, the advent of offline
meta-reinforcement learning (OMRL) has shown great promise in enabling RL
agents to multi-task and quickly adapt while acquiring knowledge safely. Among
which, Context-based OMRL (COMRL) as a popular paradigm, aims to learn a
universal policy conditioned on effective task representations. In this work,
by examining several key milestones in the field of COMRL, we propose to
integrate these seemingly independent methodologies into a unified information
theoretic framework. Most importantly, we show that the pre-existing COMRL
algorithms are essentially optimizing the same mutual information objective
between the task variable $\boldsymbol{M}$ and its latent representation
$\boldsymbol{Z}$ by implementing various approximate bounds. Based on the
theoretical insight and the information bottleneck principle, we arrive at a
novel algorithm dubbed UNICORN, which exhibits remarkable generalization across
a broad spectrum of RL benchmarks, context shift scenarios, data qualities and
deep learning architectures, attaining the new state-of-the-art. We believe
that our framework could open up avenues for new optimality bounds and COMRL
algorithms.

离线元强化学习（OMRL）的出现作为离线 RL 和元 RL 的结合在使 RL 代理能够多任务和快速适应并安全地获取知识方面显示出了巨大的潜力。我们提出将离线元强化学习的几个关键里程碑整合到一个统一的信息论框架中，并展示了现有的上下文 OMRL 算法本质上是通过实现各种近似界限优化任务变量 $oldsymbol {M}$ 与其潜在表示 $oldsymbol {Z}$ 之间的相互信息目标。基于理论见解和信息瓶颈原理，我们提出了一种新的算法称为 UNICORN，在 RL 基准、上下文转变场景、数据质量和深度学习架构的广泛领域中展示了卓越的泛化能力，达到了新的技术水平。我们相信我们的框架可以为新的优化界限和上下文 OMRL 算法开辟新的研究方向。