Semantic localization, i.e., robot self-localization with semantic image
modality, is critical in recently emerging embodied AI applications such as
point-goal navigation, object-goal navigation and vision language navigation.
However, most existing works on semantic localization focus on passive vision
tasks without viewpoint planning, or rely on additional rich modalities (e.g.,
depth measurements). Thus, the problem is largely unsolved. In this work, we
explore a lightweight, entirely CPU-based, domain-adaptive semantic
localization framework, called graph neural localizer.Our approach is inspired
by two recently emerging technologies: (1) Scene graph, which combines the
viewpoint- and appearance- invariance of local and global features; (2) Graph
neural network, which enables direct learning/recognition of graph data (i.e.,
non-vector data). Specifically, a graph convolutional neural network is first
trained as a scene graph classifier for passive vision, and then its knowledge
is transferred to a reinforcement-learning planner for active vision.
Experiments on two scenarios, self-supervised learning and unsupervised domain
adaptation, using a photo-realistic Habitat simulator validate the
effectiveness of the proposed method.

本研究提出了一个轻量级、全 CPU 基于的 “图神经本地化器” 框架，旨在解决语义定位问题。该框架结合了场景图和图神经网络技术，首先训练了一个场景图分类器用于被动视觉，然后将其知识转移到强化学习规划器用于主动视觉。实验表明，该方法在自我监督学习和无监督域适应两个场景下都具有效果。