Enabling robots to navigate following diverse language instructions in
unexplored environments is an attractive goal for human-robot interaction.
However, this goal is challenging because different navigation tasks require
different strategies. The scarcity of instruction navigation data hinders
training an instruction navigation model with varied strategies. Therefore,
previous methods are all constrained to one specific type of navigation
instruction. In this work, we propose InstructNav, a generic instruction
navigation system. InstructNav makes the first endeavor to handle various
instruction navigation tasks without any navigation training or pre-built maps.
To reach this goal, we introduce Dynamic Chain-of-Navigation (DCoN) to unify
the planning process for different types of navigation instructions.
Furthermore, we propose Multi-sourced Value Maps to model key elements in
instruction navigation so that linguistic DCoN planning can be converted into
robot actionable trajectories. With InstructNav, we complete the R2R-CE task in
a zero-shot way for the first time and outperform many task-training methods.
Besides, InstructNav also surpasses the previous SOTA method by 10.48% on the
zero-shot Habitat ObjNav and by 86.34% on demand-driven navigation DDN. Real
robot experiments on diverse indoor scenes further demonstrate our method's
robustness in coping with the environment and instruction variations.

通过引入动态导航链和价值地图，以及零样本学习的方式，提出了 InstructNav 系统来解决多种导航指令的统一规划，该系统在任务完成上优于以往方法。

InstructNav：零封闭的通用指令导航系统在未知环境中

InstructNav: Zero-shot System for Generic Instruction Navigation in  Unexplored Environment

Large language models (LLMs) are shown to possess a wealth of actionable
knowledge that can be extracted for robot manipulation in the form of reasoning
and planning. Despite the progress, most still rely on pre-defined motion
primitives to carry out the physical interactions with the environment, which
remains a major bottleneck. In this work, we aim to synthesize robot
trajectories, i.e., a dense sequence of 6-DoF end-effector waypoints, for a
large variety of manipulation tasks given an open-set of instructions and an
open-set of objects. We achieve this by first observing that LLMs excel at
inferring affordances and constraints given a free-form language instruction.
More importantly, by leveraging their code-writing capabilities, they can
interact with a visual-language model (VLM) to compose 3D value maps to ground
the knowledge into the observation space of the agent. The composed value maps
are then used in a model-based planning framework to zero-shot synthesize
closed-loop robot trajectories with robustness to dynamic perturbations. We
further demonstrate how the proposed framework can benefit from online
experiences by efficiently learning a dynamics model for scenes that involve
contact-rich interactions. We present a large-scale study of the proposed
method in both simulated and real-robot environments, showcasing the ability to
perform a large variety of everyday manipulation tasks specified in free-form
natural language. Project website: this https URL

使用大型语言模型综合视觉 - 语言模型，生成适用于各种机器人操作的闭环轨迹。