Recent developments in the field of model-based RL have proven successful in
a range of environments, especially ones where planning is essential. However,
such successes have been limited to deterministic fully-observed environments.
We present a new approach that handles stochastic and partially-observable
environments. Our key insight is to use discrete autoencoders to capture the
multiple possible effects of an action in a stochastic environment. We use a
stochastic variant of Monte Carlo tree search to plan over both the agent's
actions and the discrete latent variables representing the environment's
response. Our approach significantly outperforms an offline version of MuZero
on a stochastic interpretation of chess where the opponent is considered part
of the environment. We also show that our approach scales to DeepMind Lab, a
first-person 3D environment with large visual observations and partial
observability.

使用离散自编码器来处理动作在随机环境中引起的多种可能性，再结合随机版 Monte Carlo 树搜索算法规划代理的动作和代表环境反应的离散潜变量，明显优于 MuZero 在处理随机国际象棋和 DeepMind Lab 等部分观测模型的 RL 问题中的表现。