The study of vision-and-language navigation (VLN) has typically relied on
expert trajectories, which may not always be available in real-world situations
due to the significant effort required to collect them. On the other hand,
existing approaches to training VLN agents that go beyond available expert data
involve data augmentations or online exploration which can be tedious and
risky. In contrast, it is easy to access large repositories of suboptimal
offline trajectories. Inspired by research in offline reinforcement learning
(ORL), we introduce a new problem setup of VLN-ORL which studies VLN using
suboptimal demonstration data. We introduce a simple and effective
reward-conditioned approach that can account for dataset suboptimality for
training VLN agents, as well as benchmarks to evaluate progress and promote
research in this area. We empirically study various noise models for
characterizing dataset suboptimality among other unique challenges in VLN-ORL
and instantiate it for the VLN$\circlearrowright$BERT and MTVM architectures in
the R2R and RxR environments. Our experiments demonstrate that the proposed
reward-conditioned approach leads to significant performance improvements, even
in complex and intricate environments.

通过使用次优示范数据进行视觉语言导航 (VLN) 的离线强化学习 (ORL) 研究，引入了一种简单且有效的奖励条件方法来训练 VLN 代理，并在该领域推动研究进展和评估。实验证明，即使在复杂和复杂的环境中，所提出的奖励条件方法也能显著提高性能。