We study zero-shot instance navigation, in which the agent navigates to a
specific object without using object annotations for training. Previous object
navigation approaches apply the image-goal navigation (ImageNav) task (go to
the location of an image) for pretraining, and transfer the agent to achieve
object goals using a vision-language model. However, these approaches lead to
issues of semantic neglect, where the model fails to learn meaningful semantic
alignments. In this paper, we propose a Prioritized Semantic Learning (PSL)
method to improve the semantic understanding ability of navigation agents.
Specifically, a semantic-enhanced PSL agent is proposed and a prioritized
semantic training strategy is introduced to select goal images that exhibit
clear semantic supervision and relax the reward function from strict exact view
matching. At inference time, a semantic expansion inference scheme is designed
to preserve the same granularity level of the goal-semantic as training.
Furthermore, for the popular HM3D environment, we present an Instance
Navigation (InstanceNav) task that requires going to a specific object instance
with detailed descriptions, as opposed to the Object Navigation (ObjectNav)
task where the goal is defined merely by the object category. Our PSL agent
outperforms the previous state-of-the-art by 66% on zero-shot ObjectNav in
terms of success rate and is also superior on the new InstanceNav task. Code
will be released at this https URL science/r/PSL/.

我们提出了一种优先语义学习（PSL）方法，通过引入语义增强的 PSL 代理和优先语义训练策略来提高导航代理的语义理解能力，并设计了语义扩展推理方案以保持与训练相同的目标语义粒度级别。此外，针对流行的 HM3D 环境，我们提出了一个实例导航（InstanceNav）任务，需要根据详细描述前往特定的对象实例，与仅通过目标类别定义目标的对象导航（ObjectNav）任务不同。我们的 PSL 代理在零样本 ObjectNav 的成功率上超过了先前的最优结果 66％，并且在新的 InstanceNav 任务上也表现优越。代码将在此 https 网址上发布：science/r/PSL/。