Semantic reasoning and dynamic planning capabilities are crucial for an
autonomous agent to perform complex navigation tasks in unknown environments.
It requires a large amount of common-sense knowledge, that humans possess, to
succeed in these tasks. We present SayNav, a new approach that leverages human
knowledge from Large Language Models (LLMs) for efficient generalization to
complex navigation tasks in unknown large-scale environments. SayNav uses a
novel grounding mechanism, that incrementally builds a 3D scene graph of the
explored environment as inputs to LLMs, for generating feasible and
contextually appropriate high-level plans for navigation. The LLM-generated
plan is then executed by a pre-trained low-level planner, that treats each
planned step as a short-distance point-goal navigation sub-task. SayNav
dynamically generates step-by-step instructions during navigation and
continuously refines future steps based on newly perceived information. We
evaluate SayNav on a new multi-object navigation task, that requires the agent
to utilize a massive amount of human knowledge to efficiently search multiple
different objects in an unknown environment. SayNav outperforms an oracle based
Point-nav baseline, achieving a success rate of 95.35% (vs 56.06% for the
baseline), under the ideal settings on this task, highlighting its ability to
generate dynamic plans for successfully locating objects in large-scale new
environments.

基于大型语言模型的新方法 SayNav，通过建立探索环境的三维场景图，生成高级导航计划，并利用预训练的低级规划器逐步执行，动态生成导航过程中的指令并根据新感知信息不断改进步骤，有效在大规模新环境中定位多个不同物体。