Vision-and-Language Navigation (VLN) is a realistic but challenging task that
requires an agent to locate the target region using verbal and visual cues.
While significant advancements have been achieved recently, there are still two
broad limitations: (1) The explicit information mining for significant guiding
semantics concealed in both vision and language is still under-explored; (2)
The previously structured map method provides the average historical appearance
of visited nodes, while it ignores distinctive contributions of various images
and potent information retention in the reasoning process. This work proposes a
dual semantic-aware recurrent global-adaptive network (DSRG) to address the
above problems. First, DSRG proposes an instruction-guidance linguistic module
(IGL) and an appearance-semantics visual module (ASV) for boosting vision and
language semantic learning respectively. For the memory mechanism, a global
adaptive aggregation module (GAA) is devised for explicit panoramic observation
fusion, and a recurrent memory fusion module (RMF) is introduced to supply
implicit temporal hidden states. Extensive experimental results on the R2R and
REVERIE datasets demonstrate that our method achieves better performance than
existing methods.

该论文提出了一种双语义感知递归全局自适应网络（DSRG），包括指导语言模块和外貌 - 语义视觉模块，用于视觉和语言语义学习，以及全球自适应聚合模块和递归记忆融合模块，用于对全景观测进行显式内存融合。 虽然在 R2R 和 REVERIE 数据集上进行了广泛的实验，并证明该方法的实际性和有效性高于现有方法。