Benchmark datasets that measure camera pose accuracy have driven progress in
visual re-localisation research. To obtain poses for thousands of images, it is
common to use a reference algorithm to generate pseudo ground truth. Popular
choices include Structure-from-Motion (SfM) and
Simultaneous-Localisation-and-Mapping (SLAM) using additional sensors like
depth cameras if available. Re-localisation benchmarks thus measure how well
each method replicates the results of the reference algorithm. This begs the
question whether the choice of the reference algorithm favours a certain family
of re-localisation methods. This paper analyzes two widely used re-localisation
datasets and shows that evaluation outcomes indeed vary with the choice of the
reference algorithm. We thus question common beliefs in the re-localisation
literature, namely that learning-based scene coordinate regression outperforms
classical feature-based methods, and that RGB-D-based methods outperform
RGB-based methods. We argue that any claims on ranking re-localisation methods
should take the type of the reference algorithm, and the similarity of the
methods to the reference algorithm, into account.

本文通过分析两个广泛使用的重新本地化数据集，发现评估结果实际上随着参考算法的选择而有所差异，因此应考虑参考算法的类型和方法与参考算法的相似性。