Worldwide geolocalization aims to locate the precise location at the
coordinate level of photos taken anywhere on the Earth. It is very challenging
due to 1) the difficulty of capturing subtle location-aware visual semantics,
and 2) the heterogeneous geographical distribution of image data. As a result,
existing studies have clear limitations when scaled to a worldwide context.
They may easily confuse distant images with similar visual contents, or cannot
adapt to various locations worldwide with different amounts of relevant data.
To resolve these limitations, we propose G3, a novel framework based on
Retrieval-Augmented Generation (RAG). In particular, G3 consists of three
steps, i.e., Geo-alignment, Geo-diversification, and Geo-verification to
optimize both retrieval and generation phases of worldwide geolocalization.
During Geo-alignment, our solution jointly learns expressive multi-modal
representations for images, GPS and textual descriptions, which allows us to
capture location-aware semantics for retrieving nearby images for a given
query. During Geo-diversification, we leverage a prompt ensembling method that
is robust to inconsistent retrieval performance for different image queries.
Finally, we combine both retrieved and generated GPS candidates in
Geo-verification for location prediction. Experiments on two well-established
datasets IM2GPS3k and YFCC4k verify the superiority of G3 compared to other
state-of-the-art methods.

提出了基于检索增强生成 (RAG) 的 G3 框架，通过全球地理位置定位的几个步骤以优化检索和生成阶段，该框架在全球上下文中优于其他先进方法。

G3：基于大规模多模态模型的全球地理定位的有效和自适应框架

G3: An Effective and Adaptive Framework for Worldwide Geolocalization  Using Large Multi-Modality Models

Precise, pixel-wise geolocalization of astronaut photography is critical to
unlocking the potential of this unique type of remotely sensed Earth data,
particularly for its use in disaster management and climate change research.
Recent works have established the Astronaut Photography Localization task, but
have either proved too costly for mass deployment or generated too coarse a
localization. Thus, we present EarthMatch, an iterative homography estimation
method that produces fine-grained localization of astronaut photographs while
maintaining an emphasis on speed. We refocus the astronaut photography
benchmark, AIMS, on the geolocalization task itself, and prove our method's
efficacy on this dataset. In addition, we offer a new, fair method for image
matcher comparison, and an extensive evaluation of different matching models
within our localization pipeline. Our method will enable fast and accurate
localization of the 4.5 million and growing collection of astronaut photography
of Earth. Webpage with code and data at
this https URL

精确的像素级别的航天员摄影地理定位对于释放这种独特类型的遥感地球数据的潜力至关重要，特别是在灾害管理和气候变化研究中的应用。因此，我们提出了 EarthMatch，一种迭代单应性估计方法，可产生精细的航天员摄影地理定位，同时保持对速度的重视。