This paper focuses on training a robust RGB-D registration model without ground-truth pose supervision. Existing methods usually adopt a pairwise training strategy based on differentiable rendering, which enforces the photometric and the geometric consistency between the two registered frames as supervision. However, this frame-to-frame framework suffers from poor multi-view consistency due to factors such as lighting changes, geometry occlusion and reflective materials. In this paper, we present NeRF-UR, a novel frame-to-model optimization framework for unsupervised RGB-D registration. Instead of frame-to-frame consistency, we leverage the neural radiance field (NeRF) as a global model of the scene and use the consistency between the input and the NeRF-rerendered frames for pose optimization. This design can significantly improve the robustness in scenarios with poor multi-view consistency and provides better learning signal for the registration model. Furthermore, to bootstrap the NeRF optimization, we create a synthetic dataset, Sim-RGBD, through a photo-realistic simulator to warm up the registration model. By first training the registration model on Sim-RGBD and later unsupervisedly fine-tuning on real data, our framework enables distilling the capability of feature extraction and registration from simulation to reality. Our method outperforms the state-of-the-art counterparts on two popular indoor RGB-D datasets, ScanNet and 3DMatch. Code and models will be released for paper reproduction.

本文提出了一种无监督的RGB-D注册的新型框架NeRF-UR，通过使用神经辐射场（NeRF）作为场景的全局模型，并利用输入和NeRF重构帧之间的一致性进行姿态优化，从而显著提高了在多视角一致性差的场景中的鲁棒性，并为注册模型提供更好的学习信号。通过在合成数据集Sim-RGBD上训练注册模型，并在真实数据上进行无监督微调，使我们的框架能够将特征提取和注册的能力从模拟迁移到现实场景。我们的方法在室内RGB-D数据集ScanNet和3DMatch上优于最先进的对手。

基于NeRF引导的RGB-D配准的无监督学习