The ability to discover useful behaviours from past experience and transfer
them to new tasks is considered a core component of natural embodied
intelligence. Inspired by neuroscience, discovering behaviours that switch at
bottleneck states have been long sought after for inducing plans of minimum
description length across tasks. Prior approaches have either only supported
online, on-policy, bottleneck state discovery, limiting sample-efficiency, or
discrete state-action domains, restricting applicability. To address this, we
introduce Model-Based Offline Options (MO2), an offline hindsight framework
supporting sample-efficient bottleneck option discovery over continuous
state-action spaces. Once bottleneck options are learnt offline over source
domains, they are transferred online to improve exploration and value
estimation on the transfer domain. Our experiments show that on complex
long-horizon continuous control tasks with sparse, delayed rewards, MO2's
properties are essential and lead to performance exceeding recent option
learning methods. Additional ablations further demonstrate the impact on option
predictability and credit assignment.

本研究提出了 Model-Based Offline Options（MO2） 算法，支持在连续的状态 - 动作空间中进行样本高效的瓶颈选项发现，以提高对转移域上的探索和值估计，实验结果表明，在复杂的长视程连续控制任务中，MO2 的特性是至关重要的，并且超过了最近的选项学习方法。