In the vision-and-language navigation (VLN) task, an agent follows natural language instructions and navigate in visual environments. Compared to the indoor navigation task that has been broadly studied, navigation in real-life outdoor environments remains a significant challenge with its complicated visual inputs and an insufficient amount of instructions that illustrate the intricate urban scenes. In this paper, we introduce a Multimodal Text Style Transfer (MTST) learning approach to mitigate the problem of data scarcity in outdoor navigation tasks by effectively leveraging external multimodal resources. We first enrich the navigation data by transferring the style of the instructions generated by Google Maps API, then pre-train the navigator with the augmented external outdoor navigation dataset. Experimental results show that our MTST learning approach is model-agnostic, and our MTST approach significantly outperforms the baseline models on the outdoor VLN task, improving task completion rate by 22\% relatively on the test set and achieving new state-of-the-art performance.

本文介绍了一种多模态文本风格转换的学习方法，利用外部多模态资源缓解了户外导航任务中的数据匮乏问题，并使用Google Maps API生成的指令扩充了导航数据，进而预先训练了导航器。实验结果表明，我们的多模态文本风格转换学习方法不受模型限制，在户外视觉和语言导航任务中，相对于基准模型，改善了任务完成率8.7%。

面向户外视觉语言导航的多模态文本风格转换