Cross-modal alignment is one key challenge for Vision-and-Language Navigation
(VLN). Most existing studies concentrate on mapping the global instruction or
single sub-instruction to the corresponding trajectory. However, another
critical problem of achieving fine-grained alignment at the entity level is
seldom considered. To address this problem, we propose a novel Grounded
Entity-Landmark Adaptive (GELA) pre-training paradigm for VLN tasks. To achieve
the adaptive pre-training paradigm, we first introduce grounded entity-landmark
human annotations into the Room-to-Room (R2R) dataset, named GEL-R2R.
Additionally, we adopt three grounded entity-landmark adaptive pre-training
objectives: 1) entity phrase prediction, 2) landmark bounding box prediction,
and 3) entity-landmark semantic alignment, which explicitly supervise the
learning of fine-grained cross-modal alignment between entity phrases and
environment landmarks. Finally, we validate our model on two downstream
benchmarks: VLN with descriptive instructions (R2R) and dialogue instructions
(CVDN). The comprehensive experiments show that our GELA model achieves
state-of-the-art results on both tasks, demonstrating its effectiveness and
generalizability.

我们提出了一种新颖的基于实体 - 地标适应的预训练模型 (GELA)，用于实现细粒度的视听导航中不同模态信号的对齐。采用三个适应性预训练目标：实体短语预测，地标边界框预测和实体 - 地标语义一致性，该模型在两个任务中实现了最先进的结果，证明了其有效性和普适性。