Building on the success of Neural Radiance Fields (NeRFs), recent years have
seen significant advances in the domain of novel view synthesis. These models
capture the scene's volumetric radiance field, creating highly convincing dense
photorealistic models through the use of simple, differentiable rendering
equations. Despite their popularity, these algorithms suffer from severe
ambiguities in visual data inherent to the RGB sensor, which means that
although images generated with view synthesis can visually appear very
believable, the underlying 3D model will often be wrong. This considerably
limits the usefulness of these models in practical applications like Robotics
and Extended Reality (XR), where an accurate dense 3D reconstruction otherwise
would be of significant value. In this technical report, we present the vital
differences between view synthesis models and 3D reconstruction models. We also
comment on why a depth sensor is essential for modeling accurate geometry in
general outward-facing scenes using the current paradigm of novel view
synthesis methods. Focusing on the structure-from-motion task, we practically
demonstrate this need by extending the Plenoxel radiance field model:
Presenting an analytical differential approach for dense mapping and tracking
with radiance fields based on RGB-D data without a neural network. Our method
achieves state-of-the-art results in both the mapping and tracking tasks while
also being faster than competing neural network-based approaches.

基于 NeRFs 的成功，近年来在新颖视角合成领域取得了显著进展。然而，虽然视图合成模型在视觉上看起来很真实，但其底层的 3D 模型通常是错误的，这限制了它们在实际应用中的有效性。本技术报告介绍了视图合成模型和 3D 重建模型之间的关键区别，并指出了使用深度传感器进行准确几何建模的重要性。通过扩展 Plenoxel 辐射场模型，我们对基于 RGB-D 数据的辐射场密集建图和跟踪任务提出了一种分析微分方法，实现了优于竞争神经网络方法的最新成果，并具有更快的速度。