Semantic localization, i.e., robot self-localization with semantic image
modality, is critical in recently emerging embodied AI applications such as
point-goal navigation, object-goal navigation and vision language navigation.
However, most existing works on semantic localization focus on passive vision
tasks without viewpoint planning, or rely on additional rich modalities (e.g.,
depth measurements). Thus, the problem is largely unsolved. In this work, we
explore a lightweight, entirely CPU-based, domain-adaptive semantic
localization framework, called graph neural localizer.Our approach is inspired
by two recently emerging technologies: (1) Scene graph, which combines the
viewpoint- and appearance- invariance of local and global features; (2) Graph
neural network, which enables direct learning/recognition of graph data (i.e.,
non-vector data). Specifically, a graph convolutional neural network is first
trained as a scene graph classifier for passive vision, and then its knowledge
is transferred to a reinforcement-learning planner for active vision.
Experiments on two scenarios, self-supervised learning and unsupervised domain
adaptation, using a photo-realistic Habitat simulator validate the
effectiveness of the proposed method.

本研究提出了一个轻量级、全 CPU 基于的 “图神经本地化器” 框架，旨在解决语义定位问题。该框架结合了场景图和图神经网络技术，首先训练了一个场景图分类器用于被动视觉，然后将其知识转移到强化学习规划器用于主动视觉。实验表明，该方法在自我监督学习和无监督域适应两个场景下都具有效果。

具有图神经嵌入的主动语义定位

Active Semantic Localization with Graph Neural Embedding

In this paper we propose a novel semantic localization algorithm that
exploits multiple sensors and has precision on the order of a few centimeters.
Our approach does not require detailed knowledge about the appearance of the
world, and our maps require orders of magnitude less storage than maps utilized
by traditional geometry- and LiDAR intensity-based localizers. This is
important as self-driving cars need to operate in large environments. Towards
this goal, we formulate the problem in a Bayesian filtering framework, and
exploit lanes, traffic signs, as well as vehicle dynamics to localize robustly
with respect to a sparse semantic map. We validate the effectiveness of our
method on a new highway dataset consisting of 312km of roads. Our experiments
show that the proposed approach is able to achieve 0.05m lateral accuracy and
1.12m longitudinal accuracy on average while taking up only 0.3% of the storage
required by previous LiDAR intensity-based approaches.

本文提出了一种基于多个传感器的语义定位算法，利用车道，交通标志和车辆动态等信息，以贝叶斯滤波框架为基础，实现对稀疏语义地图的鲁棒定位，相较于传统基于几何学和 LiDAR 强度的定位器所需的储存空间降低了数个数量级，并且证明了其在 312 公里公路数据集上能够获得 0.05m 的横向精度和 1.12m 的纵向精度。