Vision-Language Navigation (VLN) is a challenging task which requires an
agent to align complex visual observations to language instructions to reach
the goal position. Most existing VLN agents directly learn to align the raw
directional features and visual features trained using one-hot labels to
linguistic instruction features. However, the big semantic gap among these
multi-modal inputs makes the alignment difficult and therefore limits the
navigation performance. In this paper, we propose Actional Atomic-Concept
Learning (AACL), which maps visual observations to actional atomic concepts for
facilitating the alignment. Specifically, an actional atomic concept is a
natural language phrase containing an atomic action and an object, e.g., ``go
up stairs''. These actional atomic concepts, which serve as the bridge between
observations and instructions, can effectively mitigate the semantic gap and
simplify the alignment. AACL contains three core components: 1) a concept
mapping module to map the observations to the actional atomic concept
representations through the VLN environment and the recently proposed
Contrastive Language-Image Pretraining (CLIP) model, 2) a concept refining
adapter to encourage more instruction-oriented object concept extraction by
re-ranking the predicted object concepts by CLIP, and 3) an observation
co-embedding module which utilizes concept representations to regularize the
observation representations. Our AACL establishes new state-of-the-art results
on both fine-grained (R2R) and high-level (REVERIE and R2R-Last) VLN
benchmarks. Moreover, the visualization shows that AACL significantly improves
the interpretability in action decision.

本文提出了 Actional Atomic-Concept Learning（AACL），它通过将视觉观测映射到动作原子概念来简化 Vision-Language Navigation 中的观测 - 指令对齐问题。AACL 在细粒度和高级别的 VLN 基准测试中均取得了新的最优结果，并且可视化结果表明 AACL 显着提高了动作决策的可解释性。