Zero-shot generalization (ZSG) to unseen dynamics is a major challenge for
creating generally capable embodied agents. To address the broader challenge,
we start with the simpler setting of contextual reinforcement learning (cRL),
assuming observability of the context values that parameterize the variation in
the system's dynamics, such as the mass or dimensions of a robot, without
making further simplifying assumptions about the observability of the Markovian
state. Toward the goal of ZSG to unseen variation in context, we propose the
contextual recurrent state-space model (cRSSM), which introduces changes to the
world model of the Dreamer (v3) (Hafner et al., 2023). This allows the world
model to incorporate context for inferring latent Markovian states from the
observations and modeling the latent dynamics. Our experiments show that such
systematic incorporation of the context improves the ZSG of the policies
trained on the ``dreams'' of the world model. We further find qualitatively
that our approach allows Dreamer to disentangle the latent state from context,
allowing it to extrapolate its dreams to the many worlds of unseen contexts.
The code for all our experiments is available at
https://github.com/sai-prasanna/dreaming_of_many_worlds.

提出了一种称为 cRSSM 的上下文循环状态空间模型，该模型通过将上下文与观察值结合起来，改进了 Dreamer 的世界模型，从而提高了在未见上下文情况下训练的策略的零 - shot 泛化能力。