Hierarchical Imitation Learning (HIL) is an effective way for robots to learn sub-skills from long-horizon unsegmented demonstrations. However, the learned hierarchical structure lacks the mechanism to transfer across multi-tasks or to new tasks, which makes them have to learn from scratch when facing a new situation. Transferring and reorganizing modular sub-skills require fast adaptation ability of the whole hierarchical structure. In this work, we propose Dual Meta Imitation Learning (DMIL), a hierarchical meta imitation learning method where the high-level network and sub-skills are iteratively meta-learned with model-agnostic meta-learning. DMIL uses the likelihood of state-action pairs from each sub-skill as the supervision for the high-level network adaptation, and use the adapted high-level network to determine different data set for each sub-skill adaptation. We theoretically prove the convergence of the iterative training process of DMIL and establish the connection between DMIL and Expectation-Maximization algorithm. Empirically, we achieve state-of-the-art few-shot imitation learning performance on the Meta-world \cite{metaworld} benchmark and competitive results on long-horizon tasks of Kitchen environments.

提出了一种名为 Dual Meta Imitation Learning(DMIL) 的层次元学习方法，使用模型无关元学习迭代元学习高级网络和子技能，并将来自每个子技能的状态操作对的似然作为高级网络适应的监督，理论上证明了 DMIL 的收敛性，并实现了在 Meta-world benchmark 中最先进的少量样本模仿学习性能和在长时间周期任务执行的 Kitchen 环境中有竞争力的结果。

双元元模仿学习传输等级结构