Neural Radiance Fields (NeRF) have recently demonstrated photo-realistic
results for the task of novel view synthesis. In this paper, we propose to
apply novel view synthesis to the robot relocalization problem: we demonstrate
improvement of camera pose regression thanks to an additional synthetic dataset
rendered by the NeRF class of algorithm. To avoid spawning novel views in
irrelevant places we selected virtual camera locations from NeRF internal
representation of the 3D geometry of the scene. We further improved
localization accuracy of pose regressors using synthesized realistic and
geometry consistent images as data augmentation during training. At the time of
publication, our approach improved state of the art with a 60% lower error on
Cambridge Landmarks and 7-scenes datasets. Hence, the resulting accuracy
becomes comparable to structure-based methods, without any architecture
modification or domain adaptation constraints. Since our method allows almost
infinite generation of training data, we investigated limitations of camera
pose regression depending on size and distribution of data used for training on
public benchmarks. We concluded that pose regression accuracy is mostly bounded
by relatively small and biased datasets rather than capacity of the pose
regression model to solve the localization task.

本文提出将新视角合成应用于机器人重定位问题，通过 NeRF 类算法所渲染的额外合成数据集，从内部表示的三维场景几何形态中选择虚拟相机位置，进一步改进了姿态回归器的定位精度，结果大幅降低了 Cambridge 地标和 7 景数据集的误差，并将精度与基于结构的方法相媲美，无需架构修改或领域适应性约束，因为该方法允许几乎无限产生训练数据，我们调查了在公共基准数据集上进行训练所使用的数据大小和分布对相机姿态回归的限制，得出结论，姿态回归精度在很大程度上受到相对较小且有偏的数据集的限制，而非姿态回归模型解决本地化任务的能力。同时，采用合成的逼真且具有几何一致性的图像进行数据增强。