In this work, we consider one-shot imitation learning for object
rearrangement tasks, where an AI agent needs to watch a single expert
demonstration and learn to perform the same task in different environments. To
achieve a strong generalization, the AI agent must infer the spatial goal
specification for the task. However, there can be multiple goal specifications
that fit the given demonstration. To address this, we propose a reward learning
approach, Graph-based Equivalence Mappings (GEM), that can discover spatial
goal representations that are aligned with the intended goal specification,
enabling successful generalization in unseen environments. Specifically, GEM
represents a spatial goal specification by a reward function conditioned on i)
a graph indicating important spatial relationships between objects and ii)
state equivalence mappings for each edge in the graph indicating invariant
properties of the corresponding relationship. GEM combines inverse
reinforcement learning and active reward learning to efficiently improve the
reward function by utilizing the graph structure and domain randomization
enabled by the equivalence mappings. We conducted experiments with simulated
oracles and with human subjects. The results show that GEM can drastically
improve the generalizability of the learned goal representations over strong
baselines.

本文讨论了一种针对物体重新排列任务的单次模仿学习方法，通过基于图等价映射的奖励学习策略发现和学习与目标规范一致的空间目标表达，并与强基线方法相比，显著提高了学习目标表达在不同环境下的泛化能力。