While recent model-free Reinforcement Learning (RL) methods have demonstrated
human-level effectiveness in gaming environments, their success in everyday
tasks like visual navigation has been limited, particularly under significant
appearance variations. This limitation arises from (i) poor sample efficiency
and (ii) over-fitting to training scenarios. To address these challenges, we
present a world model that learns invariant features using (i) contrastive
unsupervised learning and (ii) an intervention-invariant regularizer. Learning
an explicit representation of the world dynamics i.e. a world model, improves
sample efficiency while contrastive learning implicitly enforces learning of
invariant features, which improves generalization. However, the naive
integration of contrastive loss to world models fails due to a lack of
supervisory signals to the visual encoder, as world-model-based RL methods
independently optimize representation learning and agent policy. To overcome
this issue, we propose an intervention-invariant regularizer in the form of an
auxiliary task such as depth prediction, image denoising, etc., that explicitly
enforces invariance to style-interventions. Our method outperforms current
state-of-the-art model-based and model-free RL methods and significantly on
out-of-distribution point navigation task evaluated on the iGibson benchmark.
We further demonstrate that our approach, with only visual observations,
outperforms recent language-guided foundation models for point navigation,
which is essential for deployment on robots with limited computation
capabilities. Finally, we demonstrate that our proposed model excels at the
sim-to-real transfer of its perception module on Gibson benchmark.

通过对比无监督学习和介入不变正则化的使用，提出了一种学习不变特征的世界模型，以改善强化学习在视觉导航等日常任务中的限制，并在 iGibson 基准测试中取得显著的性能提升。