We introduce a novel problem, i.e., the localization of an input image within
a multi-modal reference map represented by a database of 3D scene graphs. These
graphs comprise multiple modalities, including object-level point clouds,
images, attributes, and relationships between objects, offering a lightweight
and efficient alternative to conventional methods that rely on extensive image
databases. Given the available modalities, the proposed method SceneGraphLoc
learns a fixed-sized embedding for each node (i.e., representing an object
instance) in the scene graph, enabling effective matching with the objects
visible in the input query image. This strategy significantly outperforms other
cross-modal methods, even without incorporating images into the map embeddings.
When images are leveraged, SceneGraphLoc achieves performance close to that of
state-of-the-art techniques depending on large image databases, while requiring
three orders-of-magnitude less storage and operating orders-of-magnitude
faster. The code will be made public.

这篇论文介绍了一个新问题，即在由 3D 场景图数据库表示的多模式参考地图中定位输入图像。通过学习场景图中每个节点的固定大小嵌入，该方法提供了一种轻量级且高效的替代方法，以有效地匹配输入查询图像中可见的对象实例，实现了超出其他跨模态方法的性能表现，并在使用图像时实现了与大型图像数据库依赖的最先进技术相当的性能，同时存储效率提高了三个数量级并且运行速度加快了几个数量级。