Building generalizable goal-conditioned agents from rich observations is a
key to reinforcement learning (RL) solving real world problems. Traditionally
in goal-conditioned RL, an agent is provided with the exact goal they intend to
reach. However, it is often not realistic to know the configuration of the goal
before performing a task. A more scalable framework would allow us to provide
the agent with an example of an analogous task, and have the agent then infer
what the goal should be for its current state. We propose a new form of state
abstraction called goal-conditioned bisimulation that captures functional
equivariance, allowing for the reuse of skills to achieve new goals. We learn
this representation using a metric form of this abstraction, and show its
ability to generalize to new goals in simulation manipulation tasks. Further,
we prove that this learned representation is sufficient not only for goal
conditioned tasks, but is amenable to any downstream task described by a
state-only reward function. Videos can be found at
this https URL

提出了一种名为 Goal-conditioned bisimulation 的状态抽象形式来捕捉功能等变性，该方法可以让代理器在没有明确目标的情况下从先前的行为中学习新的目标，并且证明其可以适用于任何由纯状态奖励函数描述的下游任务