Vision-and-Language Navigation (VLN), as a crucial research problem of
Embodied AI, requires an embodied agent to navigate through complex 3D
environments following natural language instructions. Recent research has
highlighted the promising capacity of large language models (LLMs) in VLN by
improving navigational reasoning accuracy and interpretability. However, their
predominant use in an offline manner usually suffers from substantial domain
gap between the VLN task and the LLM training corpus. This paper introduces a
novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill
parameter-efficient in-domain training to enable self-guided navigational
decision, leading to a significant mitigation of the domain gap in a
cost-effective manner. Specifically, at each timestep, the LLM is prompted to
forecast the navigational chain-of-thought by: 1) acting as a world model to
imagine the next observation according to the instruction, 2) selecting the
candidate observation that best aligns with the imagination, and 3) determining
the action based on the reasoning from the prior steps. Through constructing
formalized labels for training, the LLM can learn to generate desired and
reasonable chain-of-thought outputs for improving the action decision.
Experimental results across various training settings and popular VLN
benchmarks (e.g., Room-to-Room (R2R), Room-across-Room (RxR), Room-for-Room
(R4R)) show the significant superiority of NavCoT over the direct action
prediction variants. Through simple parameter-efficient finetuning, our NavCoT
outperforms a recent GPT4-based approach with ~7% relative improvement on the
R2R dataset. We believe that NavCoT will help unlock more task-adaptive and
scalable LLM-based embodied agents, which are helpful for developing real-world
robotics applications. Code is available at
this https URL

通过构建导航思维链（NavCoT）进行领域内训练的新策略，提高了基于大型语言模型（LLM）的自主导航决策性能以减少领域差距。NavCoT 相对于直接行动预测的变体在各种训练设置和流行的视听导航基准上表现明显优越，并通过简单的参数微调在 R2R 数据集上实现了约 7％的相对改进。该方法将帮助开发更适应任务且可扩展的基于 LLM 的真实世界机器人应用。