Vision-language navigation is the task of directing an embodied agent to
navigate in 3D scenes with natural language instructions. For the agent,
inferring the long-term navigation target from visual-linguistic clues is
crucial for reliable path planning, which, however, has rarely been studied
before in literature. In this article, we propose a Target-Driven Structured
Transformer Planner (TD-STP) for long-horizon goal-guided and room layout-aware
navigation. Specifically, we devise an Imaginary Scene Tokenization mechanism
for explicit estimation of the long-term target (even located in unexplored
environments). In addition, we design a Structured Transformer Planner which
elegantly incorporates the explored room layout into a neural attention
architecture for structured and global planning. Experimental results
demonstrate that our TD-STP substantially improves previous best methods'
success rate by 2% and 5% on the test set of R2R and REVERIE benchmarks,
respectively. Our code is available at this https URL .

本文提出了一种基于目标驱动的结构化 Transformer 规划器（TD-STP）用于长期目标导向和房间布局感知的视觉语言导航任务，该规划器设计了想象场景的分词机制以及一种结构化的全局规划的神经注意力架构，并在 R2R 和 REVERIE 基准测试数据集上的测试结果上比现有最优方法的成功率分别提高了 2％和 5％