In this paper, we introduce the map-language navigation task where an agent
executes natural language instructions and moves to the target position based
only on a given 3D semantic map. To tackle the task, we design the
instruction-aware Path Proposal and Discrimination model (iPPD). Our approach
leverages map information to provide instruction-aware path proposals, i.e., it
selects all potential instruction-aligned candidate paths to reduce the
solution space. Next, to represent the map observations along a path for a
better modality alignment, a novel Path Feature Encoding scheme tailored for
semantic maps is proposed. An attention-based Language Driven Discriminator is
designed to evaluate path candidates and determine the best path as the final
result. Our method can naturally avoid error accumulation compared with
single-step greedy decision methods. Comparing to a single-step imitation
learning approach, iPPD has performance gains above 17% on navigation success
and 0.18 on path matching measurement nDTW in challenging unseen environments.

本文介绍了地图 - 语言导航任务，引入了基于给定三维语义地图的自然语言指令的路径规划和区分模型 (iPPD)，并提出了适用于语义地图的新型路径特征编码方案和基于注意力机制的语言驱动鉴别器，相较于单步贪婪决策方法和单步模仿学习方法，在未知环境中具有 17% 的导航成功率和 0.18 的路径匹配度的性能提升。