The task that requires an agent to navigate to a given object through only
visual observation is called visual object navigation (VON). The main
bottlenecks of VON are strategies exploration and prior knowledge exploitation.
Traditional strategies exploration ignores the differences of searching and
navigating stages, using the same reward in two stages, which reduces
navigation performance and training efficiency. Our study enables the agent to
explore larger area in searching stage and seek the optimal path in navigating
stage, improving the success rate of navigation. Traditional prior knowledge
exploitation focused on learning and utilizing object association, which
ignored the depth and obstacle information in the environment. This paper uses
the RGB and depth information of the training scene to pretrain the feature
extractor, which improves navigation efficiency. The obstacle information is
memorized by the agent during the navigation, reducing the probability of
collision and deadlock. Depth, obstacle and other prior knowledge are
concatenated and input into the policy network, and navigation actions are
output under the training of two-stage rewards. We evaluated our method on
AI2-Thor and RoboTHOR and demonstrated that it significantly outperforms
state-of-the-art (SOTA) methods on success rate and navigation efficiency.

通过使用 RGB 和深度信息进行预训练以及在两阶段奖励训练下输出导航动作，本研究在视觉物体导航中显著提高了成功率和导航效率。

基于障碍地图的物体导航的两阶段深度增强学习

Two-Stage Depth Enhanced Learning with Obstacle Map For Object  Navigation

Visual object navigation using learning methods is one of the key tasks in
mobile robotics. This paper introduces a new representation of a scene semantic
map formed during the embodied agent interaction with the indoor environment.
It is based on a neural network method that adjusts the weights of the
segmentation model with backpropagation of the predicted fusion loss values
during inference on a regular (backward) or delayed (forward) image sequence.
We have implemented this representation into a full-fledged navigation approach
called SkillTron, which can select robot skills from end-to-end policies based
on reinforcement learning and classic map-based planning methods. The proposed
approach makes it possible to form both intermediate goals for robot
exploration and the final goal for object navigation. We conducted intensive
experiments with the proposed approach in the Habitat environment, which showed
a significant superiority in navigation quality metrics compared to
state-of-the-art approaches. The developed code and used custom datasets are
publicly available at github.com/AIRI-Institute/skill-fusion.

使用学习方法的视觉对象导航是移动机器人的关键任务之一，本文介绍了一种在与室内环境进行混合体互动过程中形成的场景语义地图的新表示方法，该方法基于神经网络通过反向或正向图像序列上的预测融合损失值的反向传播来调整分割模型的权重。我们将此表示方法实现到一种名为 SkillTron 的完整导航方法中，该方法可以根据强化学习和基于地图的经典规划方法从端到端策略中选择机器人技能。所提出的方法能够为机器人探索制定中间目标和对象导航制定最终目标，我们在 Habitat 环境中对所提出的方法进行了大量实验，在导航质量指标方面表现出明显的优势，与现有技术方法相比。所开发的代码和使用的自定义数据集可以在 github.com/AIRI-Institute/skill-fusion 上公开获取。

基于技能的视觉物体导航的交互式语义地图表示

Interactive Semantic Map Representation for Skill-based Visual Object  Navigation

The task of Visual Object Navigation (VON) involves an agent's ability to
locate a particular object within a given scene. In order to successfully
accomplish the VON task, two essential conditions must be fulfilled:1) the user
must know the name of the desired object; and 2) the user-specified object must
actually be present within the scene. To meet these conditions, a simulator can
incorporate pre-defined object names and positions into the metadata of the
scene. However, in real-world scenarios, it is often challenging to ensure that
these conditions are always met. Human in an unfamiliar environment may not
know which objects are present in the scene, or they may mistakenly specify an
object that is not actually present. Nevertheless, despite these challenges,
human may still have a demand for an object, which could potentially be
fulfilled by other objects present within the scene in an equivalent manner.
Hence, we propose Demand-driven Navigation (DDN), which leverages the user's
demand as the task instruction and prompts the agent to find the object matches
the specified demand. DDN aims to relax the stringent conditions of VON by
focusing on fulfilling the user's demand rather than relying solely on
predefined object categories or names. We propose a method first acquire
textual attribute features of objects by extracting common knowledge from a
large language model. These textual attribute features are subsequently aligned
with visual attribute features using Contrastive Language-Image Pre-training
(CLIP). By incorporating the visual attribute features as prior knowledge, we
enhance the navigation process. Experiments on AI2Thor with the ProcThor
dataset demonstrate the visual attribute features improve the agent's
navigation performance and outperform the baseline methods commonly used in
VON.

通过 Contrastive Language-Image Pre-training（CLIP） 基于视觉属性特征，我们提出了一种 Demand-driven Navigation（DDN）方法，以满足用户对指定需求的物体的导航要求，并在 AI2Thor 的 ProcThor 数据集上展示了其比常见的 VON 基准方法更好的导航性能。