Existing work on vision and language navigation mainly relies on
navigation-related losses to establish the connection between vision and
language modalities, neglecting aspects of helping the navigation agent build a
deep understanding of the visual environment. In our work, we provide indirect
supervision to the navigation agent through a hint generator that provides
detailed visual descriptions. The hint generator assists the navigation agent
in developing a global understanding of the visual environment. It directs the
agent's attention toward related navigation details, including the relevant
sub-instruction, potential challenges in recognition and ambiguities in
grounding, and the targeted viewpoint description. To train the hint generator,
we construct a synthetic dataset based on landmarks in the instructions and
visible and distinctive objects in the visual environment. We evaluate our
method on the R2R and R4R datasets and achieve state-of-the-art on several
metrics. The experimental results demonstrate that generating hints not only
enhances the navigation performance but also helps improve the interpretability
of the agent's actions.

导航和语言导航的现有研究主要依赖于导航相关的损失，以建立视觉和语言模态之间的联系，忽视了帮助导航代理机构建立对视觉环境的深入理解的方面。在我们的工作中，我们通过提示生成器为导航代理提供间接监督，提供详细的视觉描述。提示生成器帮助导航代理机构开发对视觉环境的整体理解，指导代理机构注意相关的导航细节，包括相关的子指导、识别的潜在挑战和限制、以及目标视点描述。为了训练提示生成器，我们根据指导中的地标和可见的与众不同的物体构建了一个合成数据集。我们在 R2R 和 R4R 数据集上评估了我们的方法，并在几个度量标准上取得了最先进的结果。实验结果表明，生成提示不仅提高了导航性能，还有助于提高代理机构行动的可解释性。