We introduce a novel monocular visual odometry (VO) system, NeRF-VO, that
integrates learning-based sparse visual odometry for low-latency camera
tracking and a neural radiance scene representation for sophisticated dense
reconstruction and novel view synthesis. Our system initializes camera poses
using sparse visual odometry and obtains view-dependent dense geometry priors
from a monocular depth prediction network. We harmonize the scale of poses and
dense geometry, treating them as supervisory cues to train a neural implicit
scene representation. NeRF-VO demonstrates exceptional performance in both
photometric and geometric fidelity of the scene representation by jointly
optimizing a sliding window of keyframed poses and the underlying dense
geometry, which is accomplished through training the radiance field with volume
rendering. We surpass state-of-the-art methods in pose estimation accuracy,
novel view synthesis fidelity, and dense reconstruction quality across a
variety of synthetic and real-world datasets, while achieving a higher camera
tracking frequency and consuming less GPU memory.

我们引入了一种新颖的单目视觉里程计（VO）系统 NeRF-VO，它整合了基于学习的稀疏视觉里程计用于低延迟相机跟踪和神经辐射场景表示用于复杂的密集重建和新视角合成。我们的系统使用稀疏视觉里程计初始化相机位姿，并从单目深度预测网络获取视角相关的密集几何先验。我们将位姿和密集几何的尺度统一起来，将它们视为监督信号来训练神经隐式场景表示。NeRF-VO 通过联合优化关键帧位姿和底层密集几何，在边缘项渲染的过程中训练辐射场，展现了出色的场景表示的光度和几何保真度，在各种合成和真实世界的数据集上超过了最先进的方法，在姿态估计准确性、新视角合成保真度和密集重建质量方面都取得了更好的性能，并且在相机跟踪频率更高和 GPU 内存消耗更少的情况下。