We propose a novel approach to visual navigation in unknown environments where the agent is guided by conceiving the next observations it expects to see after taking the next best action. This is achieved by learning a variational Bayesian model that generates the next expected observations (NEO) conditioned on the current observations of the agent and the target view. Our approach predicts the next best action based on the current observation and NEO. Our generative model is learned through optimizing a variational objective encompassing two key designs. First, the latent distribution is conditioned on current observations and target view, supporting model-based, target-driven navigation. Second, the latent space is modeled with a Mixture of Gaussians conditioned on the current observation and next best action. Our use of mixture-of-posteriors prior effectively alleviates the issue of over-regularized latent space, thus facilitating model generalization in novel scenes. Moreover, the NEO generation models the forward dynamics of the agent-environment interaction, which improves the quality of approximate inference and hence benefits data efficiency. We have conducted extensive evaluations on both real-world and synthetic benchmarks, and show that our model outperforms the state-of-the-art RL-based methods significantly in terms of success rate, data efficiency, and cross-scene generalization.

通过学习一个可变性贝叶斯模型来生成下一个观察期望，从而提高视觉导航的跨目标和跨场景泛化能力。我们的模型基于当前的观察和目标视图，以模型驱动和目标驱动的方式，通过混合高斯先验形成潜在空间，来有效地改善模型的泛化能力，并模拟了代理-环境交互的前向动力学，提高了近似推理的质量和数据效率。实验结果说明，我们的模型在成功率、数据效率和泛化能力方面均优于现有的最先进模型。

NeoNav: 通过生成下一个预期的观测来改善视觉导航的普适性