Visual Semantic Navigation (VSN) is the ability of a robot to learn visual
semantic information for navigating in unseen environments. These VSN models
are typically tested in those virtual environments where they are trained,
mainly using reinforcement learning based approaches. Therefore, we do not yet
have an in-depth analysis of how these models would behave in the real world.
In this work, we propose a new solution to integrate VSN models into real
robots, so that we have true embodied agents. We also release a novel ROS-based
framework for VSN, ROS4VSN, so that any VSN-model can be easily deployed in any
ROS-compatible robot and tested in a real setting. Our experiments with two
different robots, where we have embedded two state-of-the-art VSN agents,
confirm that there is a noticeable performance difference of these VSN
solutions when tested in real-world and simulation environments. We hope that
this research will endeavor to provide a foundation for addressing this
consequential issue, with the ultimate aim of advancing the performance and
efficiency of embodied agents within authentic real-world scenarios. Code to
reproduce all our experiments can be found at
this https URL

本研究提出了一种新的方法将 Visual Semantic Navigation（VSN）模型集成到真实机器人中，并针对真实环境中的性能差异进行实验验证，以期推动在真实世界情景中提升具体实体机器人的性能和效率。

实体机器人的视觉语义导航

Visual Semantic Navigation with Real Robots

In visual semantic navigation, the robot navigates to a target object with
egocentric visual observations and the class label of the target is given. It
is a meaningful task inspiring a surge of relevant research. However, most of
the existing models are only effective for single-agent navigation, and a
single agent has low efficiency and poor fault tolerance when completing more
complicated tasks. Multi-agent collaboration can improve the efficiency and has
strong application potentials. In this paper, we propose the multi-agent visual
semantic navigation, in which multiple agents collaborate with others to find
multiple target objects. It is a challenging task that requires agents to learn
reasonable collaboration strategies to perform efficient exploration under the
restrictions of communication bandwidth. We develop a hierarchical decision
framework based on semantic mapping, scene prior knowledge, and communication
mechanism to solve this task. The results of testing experiments in unseen
scenes with both known objects and unknown objects illustrate the higher
accuracy and efficiency of the proposed model compared with the single-agent
model.

本文提出了一种基于多智能体协作的视觉语义导航方法，通过分层决策框架、场景先验知识和通信机制，可以使多个机器人协同完成探索任务，并在测试实验中表现出与单一智能体模型相比更高的准确性和效率。

具备场景先验知识的多智能体具身视觉语义导航

Multi-Agent Embodied Visual Semantic Navigation with Scene Prior  Knowledge

With the rise of automation, unmanned vehicles became a hot topic both as
commercial products and as a scientific research topic. It composes a
multi-disciplinary field of robotics that encompasses embedded systems, control
theory, path planning, Simultaneous Localization and Mapping (SLAM), scene
reconstruction, and pattern recognition. In this work, we present our
exploratory research of how sensor data fusion and state-of-the-art machine
learning algorithms can perform the Embodied Artificial Intelligence (E-AI)
task called Visual Semantic Navigation. This task, a.k.a Object-Goal Navigation
(ObjectNav) consists of autonomous navigation using egocentric visual
observations to reach an object belonging to the target semantic class without
prior knowledge of the environment. Our method reached fourth place on the
Habitat Challenge 2021 ObjectNav on the Minival phase and the Test-Standard
Phase.

本文探究了如何利用多传感器数据融合和最先进的机器学习算法，实现名为视觉语义导航的任务，在不需要先验环境知识的情况下使用自我中心视觉观测来到达属于目标语义类别的物体。我们的方法在 Habitat Challenge 2021 ObjectNav 的 Minival 阶段和 Test-Standard 阶段中，取得了第四名的成绩。

超越观察：一种 ObjectNav 的方法

BEyond observation: an approach for ObjectNav

This paper focuses on visual semantic navigation, the task of producing
actions for an active agent to navigate to a specified target object category
in an unknown environment. To complete this task, the algorithm should
simultaneously locate and navigate to an instance of the category. In
comparison to the traditional point goal navigation, this task requires the
agent to have a stronger contextual prior to indoor environments. We introduce
SSCNav, an algorithm that explicitly models scene priors using a
confidence-aware semantic scene completion module to complete the scene and
guide the agent's navigation planning. Given a partial observation of the
environment, SSCNav first infers a complete scene representation with semantic
labels for the unobserved scene together with a confidence map associated with
its own prediction. Then, a policy network infers the action from the scene
completion result and confidence map. Our experiments demonstrate that the
proposed scene completion module improves the efficiency of the downstream
navigation policies. Video, code, and data: this https URL

探讨了通过使用语义场景完整模块来辅助导航规划的任务，这个模块可以同时定位和导航到所需要的物体，提高了导航策略的效率。