This work exploits action equivariance for representation learning in
reinforcement learning. Equivariance under actions states that transitions in
the input space are mirrored by equivalent transitions in latent space, while
the map and transition functions should also commute. We introduce a
contrastive loss function that enforces action equivariance on the learned
representations. We prove that when our loss is zero, we have a homomorphism of
a deterministic Markov Decision Process (MDP). Learning equivariant maps leads
to structured latent spaces, allowing us to build a model on which we plan
through value iteration. We show experimentally that for deterministic MDPs,
the optimal policy in the abstract MDP can be successfully lifted to the
original MDP. Moreover, the approach easily adapts to changes in the goal
states. Empirically, we show that in such MDPs, we obtain better
representations in fewer epochs compared to representation learning approaches
using reconstructions, while generalizing better to new goals than model-free
approaches.

该研究利用行动等变性原理进行表示学习，提出了一种对学习到的表示进行行动等变性限制的对比损失函数并证明了当损失函数为零时的决策过程是同态的。该方法能够获得效果更好的表示并具有更好的泛化能力。