Scene coordinates regression (SCR), i.e., predicting 3D coordinates for every
pixel of a given image, has recently shown promising potential. However,
existing methods remain mostly scene-specific or limited to small scenes and
thus hardly scale to realistic datasets. In this paper, we propose a new
paradigm where a single generic SCR model is trained once to be then deployed
to new test scenes, regardless of their scale and without further finetuning.
For a given query image, it collects inputs from off-the-shelf image retrieval
techniques and Structure-from-Motion databases: a list of relevant database
images with sparse pointwise 2D-3D annotations. The model is based on the
transformer architecture and can take a variable number of images and sparse
2D-3D annotations as input. It is trained on a few diverse datasets and
significantly outperforms other scene regression approaches on several
benchmarks, including scene-specific models, for visual localization. In
particular, we set a new state of the art on the Cambridge localization
benchmark, even outperforming feature-matching-based approaches.

在这篇论文中，我们提出了一种新的范式，即训练一个单一的通用场景坐标回归模型，然后将其部署到新的测试场景中，无论其规模如何，也无需进一步微调。该模型基于转换器架构，可以接受可变数量的图像和稀疏的 2D-3D 注释作为输入，并在一些多样的数据集上进行训练，显著优于其他场景回归方法，包括针对视觉定位的特定场景模型，并在剑桥定位基准测试中创造了新的最佳表现，甚至超过基于特征匹配的方法。