Persons with visual impairments (PwVI) have difficulties understanding and
navigating spaces around them. Current wayfinding technologies either focus
solely on navigation or provide limited communication about the environment.
Motivated by recent advances in visual-language grounding and semantic
navigation, we propose DRAGON, a guiding robot powered by a dialogue system and
the ability to associate the environment with natural language. By
understanding the commands from the user, DRAGON is able to guide the user to
the desired landmarks on the map, describe the environment, and answer
questions from visual observations. Through effective utilization of dialogue,
the robot can ground the user's free-form descriptions to landmarks in the
environment, and give the user semantic information through spoken language. We
conduct a user study with blindfolded participants in an everyday indoor
environment. Our results demonstrate that DRAGON is able to communicate with
the user smoothly, provide a good guiding experience, and connect users with
their surrounding environment in an intuitive manner.

提出了 DRAGON，一种由对话系统驱动的导航机器人，通过理解用户的命令，DRAGON 能够引导用户到地图上所需的地标，描述环境，并回答来自视觉观察的问题。通过对话的有效利用，机器人可以将用户的自由形式描述与环境中的地标联系起来，并通过口语提供语义信息。在日常室内环境中开展了一个盲目的用户研究，结果表明 DRAGON 能够与用户顺畅交流，提供良好的导航体验，并以直观的方式连接用户与周围环境。

基于对话的机器人辅助导航系统：视觉语言基础

DRAGON: A Dialogue-Based Robot for Assistive Navigation with Visual  Language Grounding

How do humans navigate to target objects in novel scenes? Do we use the
semantic/functional priors we have built over years to efficiently search and
navigate? For example, to search for mugs, we search cabinets near the coffee
machine and for fruits we try the fridge. In this work, we focus on
incorporating semantic priors in the task of semantic navigation. We propose to
use Graph Convolutional Networks for incorporating the prior knowledge into a
deep reinforcement learning framework. The agent uses the features from the
knowledge graph to predict the actions. For evaluation, we use the AI2-THOR
framework. Our experiments show how semantic knowledge improves performance
significantly. More importantly, we show improvement in generalization to
unseen scenes and/or objects. The supplementary video can be accessed at the
following link: this https URL .

本文通过使用图卷积网络和深度强化学习框架将语义先验知识纳入到语义导航的任务中，证明语义知识如何显著提高性能并改善对未知场景和 / 或对象的泛化能力。