Vision-and-Language Navigation (VLN) aims to develop intelligent agents to
navigate in unseen environments only through language and vision supervision.
In the recently proposed continuous settings (continuous VLN), the agent must
act in a free 3D space and faces tougher challenges like real-time execution,
complex instruction understanding, and long action sequence prediction. For a
better performance in continuous VLN, we design a multi-level instruction
understanding procedure and propose a novel model, Multi-Level Attention
Network (MLANet). The first step of MLANet is to generate sub-instructions
efficiently. We design a Fast Sub-instruction Algorithm (FSA) to segment the
raw instruction into sub-instructions and generate a new sub-instruction
dataset named ``FSASub". FSA is annotation-free and faster than the current
method by 70 times, thus fitting the real-time requirement in continuous VLN.
To solve the complex instruction understanding problem, MLANet needs a global
perception of the instruction and observations. We propose a Multi-Level
Attention (MLA) module to fuse vision, low-level semantics, and high-level
semantics, which produce features containing a dynamic and global comprehension
of the task. MLA also mitigates the adverse effects of noise words, thus
ensuring a robust understanding of the instruction. To correctly predict
actions in long trajectories, MLANet needs to focus on what sub-instruction is
being executed every step. We propose a Peak Attention Loss (PAL) to improve
the flexible and adaptive selection of the current sub-instruction. PAL
benefits the navigation agent by concentrating its attention on the local
information, thus helping the agent predict the most appropriate actions. We
train and test MLANet in the standard benchmark. Experiment results show MLANet
outperforms baselines by a significant margin.

为了更好地在连续的虚拟现实环境中实现语音导航，本文提出了一个多层次的指令理解机制和一个名为 MLANet 的新模型，它包含子指令生成的快速算法 (FSA)、多层次关注模块 (MLA) 来调和指令和视觉信号，以及用于提高对当前子指令选择能力的峰值关注损失 (PAL)，实验结果表明 MLANet 优于基线模型。