We study the image-based geolocalization problem that aims to locate
ground-view query images on cartographic maps. Previous methods often utilize
cross-view localization techniques to match ground-view query images with 2D
maps. However, the performance of these methods is frequently unsatisfactory
due to the significant cross-view appearance differences. In this paper, we
extend cross-view matching to 2.5D spaces, where the heights of the structures
- such as trees, buildings, and other objects - can provide additional
information to guide the cross-view matching. We present a new approach to
learning representative embeddings from multi-model data. Specifically, we
first align 2D maps to ground-view panoramic images with polar transform to
reduce the gap between panoramic images and maps. Then we leverage global
fusion to fuse the multi-modal features from 2D and 2.5D maps to increase the
distinctiveness of location embeddings. We construct the first large-scale
ground-to-2.5D map geolocalization dataset to validate our method and
facilitate the research. We test our learned embeddings on two popular
localization approaches, i.e., single-image based localization, and route based
localization. Extensive experiments demonstrate that our proposed method
achieves significantly higher localization accuracy and faster convergence than
previous 2D map-based approaches.

本文研究基于图像的地理定位问题，通过在地图上确定地面视角的查询图像。我们提出了一种新的方法，通过 2.5D 空间中的结构高度来引导跨视图匹配，并借助多模态数据学习代表性嵌入。我们构建了第一个大规模的地面到 2.5D 地图地理定位数据集来验证我们的方法，并在单图像定位和路径定位等任务上进行了广泛实验，结果表明我们的方法在定位精度和收敛速度上明显优于之前的基于 2D 地图的方法。