Skill-based reinforcement learning (RL) approaches have shown considerable
promise, especially in solving long-horizon tasks via hierarchical structures.
These skills, learned task-agnostically from offline datasets, can accelerate
the policy learning process for new tasks. Yet, the application of these skills
in different domains remains restricted due to their inherent dependency on the
datasets, which poses a challenge when attempting to learn a skill-based policy
via RL for a target domain different from the datasets' domains. In this paper,
we present a novel offline skill learning framework DuSkill which employs a
guided Diffusion model to generate versatile skills extended from the limited
skills in datasets, thereby enhancing the robustness of policy learning for
tasks in different domains. Specifically, we devise a guided diffusion-based
skill decoder in conjunction with the hierarchical encoding to disentangle the
skill embedding space into two distinct representations, one for encapsulating
domain-invariant behaviors and the other for delineating the factors that
induce domain variations in the behaviors. Our DuSkill framework enhances the
diversity of skills learned offline, thus enabling to accelerate the learning
procedure of high-level policies for different domains. Through experiments, we
show that DuSkill outperforms other skill-based imitation learning and RL
algorithms for several long-horizon tasks, demonstrating its benefits in
few-shot imitation and online RL.

通过离线数据集学习的、能够在不同领域中应用的多功能技能是一项全新的离线技能学习框架 DuSkill 的核心，通过引导式扩散模型生成可以应用于任务的多功能技能，从而增加不同领域中策略学习的稳健性。