This paper describes a multi-modal data association method for global
localization using object-based maps and camera images. In global localization,
or relocalization, using object-based maps, existing methods typically resort
to matching all possible combinations of detected objects and landmarks with
the same object category, followed by inlier extraction using RANSAC or
brute-force search. This approach becomes infeasible as the number of landmarks
increases due to the exponential growth of correspondence candidates. In this
paper, we propose labeling landmarks with natural language descriptions and
extracting correspondences based on conceptual similarity with image
observations using a Vision Language Model (VLM). By leveraging detailed text
information, our approach efficiently extracts correspondences compared to
methods using only object categories. Through experiments, we demonstrate that
the proposed method enables more accurate global localization with fewer
iterations compared to baseline methods, exhibiting its efficiency.

通过使用多模态数据关联方法，结合基于对象的地图和相机图像，本文描述了一种全局定位的方法。我们尝试使用 Vision Language Model (VLM)，通过自然语言描述标记地标，并基于图像观察的概念相似性提取对应关系，从而提高了全局定位的准确性和效率。