Deep Learning has revolutionized our ability to solve complex problems such
as Vision-and-Language Navigation (VLN). This task requires the agent to
navigate to a goal purely based on visual sensory inputs given natural language
instructions. However, prior works formulate the problem as a navigation graph
with a discrete action space. In this work, we lift the agent off the
navigation graph and propose a more complex VLN setting in continuous 3D
reconstructed environments. Our proposed setting, Robo-VLN, more closely mimics
the challenges of real world navigation. Robo-VLN tasks have longer trajectory
lengths, continuous action spaces, and challenges such as obstacles. We provide
a suite of baselines inspired by state-of-the-art works in discrete VLN and
show that they are less effective at this task. We further propose that
decomposing the task into specialized high- and low-level policies can more
effectively tackle this task. With extensive experiments, we show that by using
layered decision making, modularized training, and decoupling reasoning and
imitation, our proposed Hierarchical Cross-Modal (HCM) agent outperforms
existing baselines in all key metrics and sets a new benchmark for Robo-VLN.

该研究论文提出了一种基于连续三维重建环境的机器人视觉和语言导航（Robo-VLN）的设定，并利用分层决策，模块化训练和分离推理和模仿的策略，成功地解决了 Robo-VLN 这一新任务。通过使用 HCM 代理，研究者展示了比现有基线效果更好的结果，为 Robo-VLN 创造了新的基准。