We propose a new method, called PiZero, that gives an agent the ability to
plan in an abstract search space of its own creation that is completely
decoupled from the real environment. Unlike prior approaches, this enables the
agent to perform high-level planning at arbitrary timescales and reason in
terms of compound or temporally-extended actions, which can be useful in
environments where large numbers of base-level micro-actions are needed to
perform relevant macro-actions. In addition, our method is more general than
comparable prior methods because it handles settings with continuous action
spaces and partial observability. We evaluate our method on multiple domains,
including navigation tasks and Sokoban. Experimentally, it outperforms
comparable prior methods without assuming access to an environment simulator.

我们提出了一种名为 PiZero 的新方法，使代理能够在完全与真实环境脱节的自主创建的抽象搜索空间中进行规划。与之前的方法不同，这使得代理能够在任意时间尺度上进行高层规划，并以复合或时间扩展的动作形式进行推理，这在需要执行大量基础微动作来执行相关宏动作的环境中非常有用。此外，我们的方法比可比较的之前方法更通用，因为它处理具有连续动作空间和部分可观察性的设置。我们在多个领域，包括导航任务和 Sokoban，对我们的方法进行了评估。实验结果表明，我们的方法在没有假设访问环境模拟器的情况下，优于可比较的之前方法。