We introduce PIGEON, a multi-task end-to-end system for planet-scale image
geolocalization that achieves state-of-the-art performance on both external
benchmarks and in human evaluation. Our work incorporates semantic geocell
creation with label smoothing, conducts pretraining of a vision transformer on
images with geographic information, and refines location predictions with
ProtoNets across a candidate set of geocells. The contributions of PIGEON are
three-fold: first, we design a semantic geocells creation and splitting
algorithm based on open-source data which can be adapted to any geospatial
dataset. Second, we show the effectiveness of intra-geocell refinement and the
applicability of unsupervised clustering and ProtNets to the task. Finally, we
make our pre-trained CLIP transformer model, StreetCLIP, publicly available for
use in adjacent domains with applications to fighting climate change and urban
and rural scene understanding.

PIGEON 是一个多任务一体化系统，通过基于语义地理单元的创建和分割算法、vision transformer 的预训练以及在候选地理单元集合上使用 ProtoNets 优化位置预测等方式实现行星级图像地理定位的最先进性能，并公开 StreetCLIP 模型以用于应对气候变化和城乡场景理解等临近领域应用。