Vision-and-Language Navigation in Continuous Environments (VLN-CE) is one of
the most intuitive yet challenging embodied AI tasks. Agents are tasked to
navigate towards a target goal by executing a set of low-level actions,
following a series of natural language instructions. All VLN-CE methods in the
literature assume that language instructions are exact. However, in practice,
instructions given by humans can contain errors when describing a spatial
environment due to inaccurate memory or confusion. Current VLN-CE benchmarks do
not address this scenario, making the state-of-the-art methods in VLN-CE
fragile in the presence of erroneous instructions from human users. For the
first time, we propose a novel benchmark dataset that introduces various types
of instruction errors considering potential human causes. This benchmark
provides valuable insight into the robustness of VLN systems in continuous
environments. We observe a noticeable performance drop (up to -25%) in Success
Rate when evaluating the state-of-the-art VLN-CE methods on our benchmark.
Moreover, we formally define the task of Instruction Error Detection and
Localization, and establish an evaluation protocol on top of our benchmark
dataset. We also propose an effective method, based on a cross-modal
transformer architecture, that achieves the best performance in error detection
and localization, compared to baselines. Surprisingly, our proposed method has
revealed errors in the validation set of the two commonly used datasets for
VLN-CE, i.e., R2R-CE and RxR-CE, demonstrating the utility of our technique in
other tasks. Code and dataset will be made available upon acceptance at
this https URL

在连续环境中的视觉与语言导航 (VLN-CE) 是一项直观但具有挑战性的 AI 任务，我们提出了一个新颖的基准数据集，引入了各种类型的指令错误并考虑了潜在的人为原因，从而为连续环境中的 VLN 系统的鲁棒性提供了宝贵的见解。我们还正式定义了指令错误检测和定位的任务，并在我们的基准数据集上建立了一个评估协议，我们提出的基于跨模态变换器架构的有效方法在错误检测和定位方面表现最好。

注意错误！视觉与语言导航中的指示错误检测和定位

Mind the Error! Detection and Localization of Instruction Errors in  Vision-and-Language Navigation

Vision-language navigation is a task that requires an agent to follow
instructions to navigate in environments. It becomes increasingly crucial in
the field of embodied AI, with potential applications in autonomous navigation,
search and rescue, and human-robot interaction. In this paper, we propose to
address a more practical yet challenging counterpart setting - vision-language
navigation in continuous environments (VLN-CE). To develop a robust VLN-CE
agent, we propose a new navigation framework, ETPNav, which focuses on two
critical skills: 1) the capability to abstract environments and generate
long-range navigation plans, and 2) the ability of obstacle-avoiding control in
continuous environments. ETPNav performs online topological mapping of
environments by self-organizing predicted waypoints along a traversed path,
without prior environmental experience. It privileges the agent to break down
the navigation procedure into high-level planning and low-level control.
Concurrently, ETPNav utilizes a transformer-based cross-modal planner to
generate navigation plans based on topological maps and instructions. The plan
is then performed through an obstacle-avoiding controller that leverages a
trial-and-error heuristic to prevent navigation from getting stuck in
obstacles. Experimental results demonstrate the effectiveness of the proposed
method. ETPNav yields more than 10% and 20% improvements over prior
state-of-the-art on R2R-CE and RxR-CE datasets, respectively. Our code is
available at this https URL

本研究提出了 EPTNav 探索实现连续环境下视觉语言导航的方法和框架，该框架着重于生成长距离导航计划和在连续环境中进行避障控制，并利用拓扑地图和指令生成自主导航计划。实验证明该方法的有效性，并在 R2R-CE 和 RxR-CE 数据集上分别获得超过 10% 和 20% 的改进。

ETPNav：连续环境下视觉语言导航的进化拓扑规划

ETPNav: Evolving Topological Planning for Vision-Language Navigation in  Continuous Environments

This paper presents a convex-QCQP based novel path planning algorithm named
ellipsoidal constrained agent navigation (ECAN), for a challenging problem of
online path planning in completely unknown and unseen continuous environments.
ECAN plans path for the agent by making a tunnel of overlapping ellipsoids, in
an online fashion, through the environment. Convex constraints in the
ellipsoid-formation step circumvent collision with the obstacles. The problem
of online-tunneling is solved as a convex-QCQP. This paper assumes no
constraints on shape of the agent and the obstacles. However, to make the
approach clearer, this paper first introduces the framework for a point-mass
agent with point-size obstacles. After explaining the underlying principle in
drawing an ellipsoid tunnel, the framework is extended to the agent and
obstacles having finite area (2d space) and finite-volume (3d-space).

在完全未知和看不见的连续环境中，本文提出了一个名为 ellipsoidal constrained agent navigation (ECAN) 的凸 QCQP 基础路径规划算法，通过在线方式在环境中通过重叠的椭球形建立通道规划出智能体的路径，通过椭球形的形成步骤的凸约束来避免与障碍物碰撞的挑战问题。