Vision-and-Language Navigation (VLN) is a natural language grounding task
where an agent learns to follow language instructions and navigate to specified
destinations in real-world environments. A key challenge is to recognize and
stop at the correct location, especially for complicated outdoor environments.
Existing methods treat the STOP action equally as other actions, which results
in undesirable behaviors that the agent often fails to stop at the destination
even though it might be on the right path. Therefore, we propose Learning to
Stop (L2Stop), a simple yet effective policy module that differentiates STOP
and other actions. Our approach achieves the new state of the art on a
challenging urban VLN dataset Touchdown, outperforming the baseline by 6.89%
(absolute improvement) on Success weighted by Edit Distance (SED).

本论文提出了一种基于训练停止策略模块的 Vision-and-Language Navigation 模型，能够在现实世界的环境中通过自然语言指示来准确导航， 相较于传统方法性能得到了 6.89% 的提升。

学会停止：一种简单而有效的城市视觉语言导航方法

Learning to Stop: A Simple yet Effective Approach to Urban  Vision-Language Navigation

Grounding natural language in images, such as localizing "the black dog on
the left of the tree", is one of the core problems in artificial intelligence,
as it needs to comprehend the fine-grained and compositional language space.
However, existing solutions rely on the association between the holistic
language features and visual features, while neglect the nature of
compositional reasoning implied in the language. In this paper, we propose a
natural language grounding model that can automatically compose a binary tree
structure for parsing the language and then perform visual reasoning along the
tree in a bottom-up fashion. We call our model RVG-TREE: Recursive Grounding
Tree, which is inspired by the intuition that any language expression can be
recursively decomposed into two constituent parts, and the grounding confidence
score can be recursively accumulated by calculating their grounding scores
returned by sub-trees. RVG-TREE can be trained end-to-end by using the
Straight-Through Gumbel-Softmax estimator that allows the gradients from the
continuous score functions passing through the discrete tree construction.
Experiments on several benchmarks show that our model achieves the
state-of-the-art performance with more explainable reasoning.

本文提出了一种基于 RVG-TREE 的自然语言基础模型，其可以自动组成二叉树结构进行语言解析，并沿着树进行自底向上的视觉推理，实现更具解释性的推理效果。

学习使用语言树结构合成和推理视觉基础

Learning to Compose and Reason with Language Tree Structures for Visual  Grounding

Vision-and-Language Navigation (VLN) is a natural language grounding task
where agents have to interpret natural language instructions in the context of
visual scenes in a dynamic environment to achieve prescribed navigation goals.
Successful agents must have the ability to parse natural language of varying
linguistic styles, ground them in potentially unfamiliar scenes, plan and react
with ambiguous environmental feedback. Generalization ability is limited by the
amount of human annotated data. In particular, \emph{paired} vision-language
sequence data is expensive to collect. We develop a discriminator that
evaluates how well an instruction explains a given path in VLN task using
multi-modal alignment. Our study reveals that only a small fraction of the
high-quality augmented data from \citet{Fried:2018:Speaker}, as scored by our
discriminator, is useful for training VLN agents with similar performance on
previously unseen environments. We also show that a VLN agent warm-started with
pre-trained components from the discriminator outperforms the benchmark success
rates of 35.5 by 10\% relative measure on previously unseen environments.

本论文提出了一种使用多模态对齐的鉴别器来评估视觉和语言导航任务中的自然语言指令，以及通过预训练组件暖启动 VLN 代理以改善其在以前看不见的环境中的表现的方法。

视觉语言导航的多模态判别模型

Multi-modal Discriminative Model for Vision-and-Language Navigation

Humans effortlessly "program" one another by communicating goals and desires
in natural language. In contrast, humans program robotic behaviours by
indicating desired object locations and poses to be achieved, by providing RGB
images of goal configurations, or supplying a demonstration to be imitated.
None of these methods generalize across environment variations, and they convey
the goal in awkward technical terms. This work proposes joint learning of
natural language grounding and instructable behavioural policies reinforced by
perceptual detectors of natural language expressions, grounded to the sensory
inputs of the robotic agent. Our supervision is narrated visual
demonstrations(NVD), which are visual demonstrations paired with verbal
narration (as opposed to being silent). We introduce a dataset of NVD where
teachers perform activities while describing them in detail. We map the
teachers' descriptions to perceptual reward detectors, and use them to train
corresponding behavioural policies in simulation.We empirically show that our
instructable agents (i) learn visual reward detectors using a small number of
examples by exploiting hard negative mined configurations from demonstration
dynamics, (ii) develop pick-and place policies using learned visual reward
detectors, (iii) benefit from object-factorized state representations that
mimic the syntactic structure of natural language goal expressions, and (iv)
can execute behaviours that involve novel objects in novel locations at test
time, instructed by natural language.

该研究提出了一种联合学习自然语言基础和可教授行为策略的方法，并使用叙述视觉演示（NVD）对其进行监督。通过将指导者描述映射到感知奖励探测器，训练对应的行为策略。通过实验表明，具有自然语言指导的可教授代理人可以在测试时执行涉及新对象和新位置的行为。