Vision-and-language navigation requires an agent to navigate through a real
3D environment following natural language instructions. Despite significant
advances, few previous works are able to fully utilize the strong
correspondence between the visual and textual sequences. Meanwhile, due to the
lack of intermediate supervision, the agent's performance at following each
part of the instruction cannot be assessed during navigation. In this work, we
focus on the granularity of the visual and language sequences as well as the
traceability of agents through the completion of an instruction. We provide
agents with fine-grained annotations during training and find that they are
able to follow the instruction better and have a higher chance of reaching the
target at test time. We enrich the benchmark dataset Room-to-Room (R2R) with
sub-instructions and their corresponding paths. To make use of this data, we
propose effective sub-instruction attention and shifting modules that select
and attend to a single sub-instruction at each time-step. We implement our
sub-instruction modules in four state-of-the-art agents, compare with their
baseline models, and show that our proposed method improves the performance of
all four agents.
We release the Fine-Grained R2R dataset (FGR2R) and the code at
this https URL

本文提出了一个精细度更高的注释，以便在自然语言指令下更好地完成环境中的导航，为此使用了带有细致的注释的 Room-to-Room（R2R）基准数据集。并引入有效的子指令注意力和移位模块，在每个时间步选择并关注单个子指令，实现了此子指令模块，与四个最新的代理基线模型进行了比较，并展示了我们提出的方法改进了所有四个代理的性能。