We present Value Propagation (VProp), a parameter-efficient differentiable planning module built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments. Furthermore, we show that the module enables learning to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems. We evaluate on static and dynamic configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes, and on a StarCraft navigation scenario, with more complex dynamics, and pixels as input.

本文介绍了 Value Propagation（VProp），它是一组基于可微分的价值迭代的参数高效的规划模块，通过强化学习可以成功地解决未知任务，具有在更大的地图尺寸上泛化的能力，并且可以学习在动态环境下导航。使用这些模块能够提供一种成本效益高低级别、尺寸无关的规划器，适用于各种交互式导航问题。我们在 MazeBase 网格世界的静态和动态配置上进行了评估，这些世界具有不同尺寸的随机生成的环境，并且在一个更具复杂动态性，以图像像素作为输入的 StarCraft 导航情景上进行了评估。

价值传播网络