While recent model-free Reinforcement Learning (RL) methods have demonstrated
human-level effectiveness in gaming environments, their success in everyday
tasks like visual navigation has been limited, particularly under significant
appearance variations. This limitation arises from (i) poor sample efficiency
and (ii) over-fitting to training scenarios. To address these challenges, we
present a world model that learns invariant features using (i) contrastive
unsupervised learning and (ii) an intervention-invariant regularizer. Learning
an explicit representation of the world dynamics i.e. a world model, improves
sample efficiency while contrastive learning implicitly enforces learning of
invariant features, which improves generalization. However, the naive
integration of contrastive loss to world models fails due to a lack of
supervisory signals to the visual encoder, as world-model-based RL methods
independently optimize representation learning and agent policy. To overcome
this issue, we propose an intervention-invariant regularizer in the form of an
auxiliary task such as depth prediction, image denoising, etc., that explicitly
enforces invariance to style-interventions. Our method outperforms current
state-of-the-art model-based and model-free RL methods and significantly on
out-of-distribution point navigation task evaluated on the iGibson benchmark.
We further demonstrate that our approach, with only visual observations,
outperforms recent language-guided foundation models for point navigation,
which is essential for deployment on robots with limited computation
capabilities. Finally, we demonstrate that our proposed model excels at the
sim-to-real transfer of its perception module on Gibson benchmark.

通过对比无监督学习和介入不变正则化的使用，提出了一种学习不变特征的世界模型，以改善强化学习在视觉导航等日常任务中的限制，并在 iGibson 基准测试中取得显著的性能提升。

ReCoRe：世界模型的正则化对比表示学习

ReCoRe: Regularized Contrastive Representation Learning of World Model

Contrastive unsupervised learning has recently shown encouraging progress,
e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the
effectiveness of two of SimCLR's design improvements by implementing them in
the MoCo framework. With simple modifications to MoCo---namely, using an MLP
projection head and more data augmentation---we establish stronger baselines
that outperform SimCLR and do not require large training batches. We hope this
will make state-of-the-art unsupervised learning research more accessible. Code
will be made public.

通过在 MoCo 框架中实现 SimCLR 的两个设计改进（使用 MLP 投影头和更多数据增强），我们建立了更强的基线模型，而不需要大的训练批次，从而检验了对比无监督学习的有效性。