Existing approaches for vision-and-language navigation (VLN) are mainly based on cross-modal reasoning over discrete views. However, this scheme may hamper an agent's spatial and numerical reasoning because of incomplete objects within a single view and duplicate observations across views. A potential solution is mapping discrete views into a unified birds's-eye view, which can aggregate partial and duplicate observations. Existing metric maps could achieve this goal, but they suffer from less expressive semantics (e.g. usually predefined labels) and limited map size, which weakens an agent's language grounding and long-term planning ability. Inspired by the robotics community, we introduce hybrid topo-metric maps into VLN, where a topological map is used for long-term planning and a metric map for short-term reasoning. Beyond mapping with more expressive deep features, we further design a pre-training framework via the hybrid map to learn language-informed map representations, which enhances cross-modal grounding and facilitates the final language-guided navigation goal. Extensive experiments demonstrate the effectiveness of the map-based route for VLN, and the proposed method sets the new state-of-the-art on three VLN benchmarks.

本研究提出采用混合拓扑度量图来实现视觉和语言导航，其中拓扑地图用于长期规划，度量地图用于短期推理，通过深度特征和预训练框架学习语言信息地图表示，增强了跨模态接地和促进最终的语言引导导航目标，实验证明了地图路线在VLN中的有效性，该方法在三个VLN基准测试中取得了最新的最佳效果。

BEVBert: 基于拓扑度量图的语言引导导航预训练