Learning policies which are robust to changes in the environment are critical
for real world deployment of Reinforcement Learning agents. They are also
necessary for achieving good generalization across environment shifts. We focus
on bisimulation metrics, which provide a powerful means for abstracting task
relevant components of the observation and learning a succinct representation
space for training the agent using reinforcement learning. In this work, we
extend the bisimulation framework to also account for context dependent
observation shifts. Specifically, we focus on the simulator based learning
setting and use alternate observations to learn a representation space which is
invariant to observation shifts using a novel bisimulation based objective.
This allows us to deploy the agent to varying observation settings during test
time and generalize to unseen scenarios. We further provide novel theoretical
bounds for simulator fidelity and performance transfer guarantees for using a
learnt policy to unseen shifts. Empirical analysis on the high-dimensional
image based control domains demonstrates the efficacy of our method.

本文提出了一种基于 bisimulation 的模拟器学习方法，在考虑环境变化的情况下，使用备选观测来学习一个对观测变换不变的表示空间，以实现对测试阶段环境的适应，并在高维图像控制领域上进行实证分析。