Online reconstructing and rendering of large-scale indoor scenes is a
long-standing challenge. SLAM-based methods can reconstruct 3D scene geometry
progressively in real time but can not render photorealistic results. While
NeRF-based methods produce promising novel view synthesis results, their long
offline optimization time and lack of geometric constraints pose challenges to
efficiently handling online input. Inspired by the complementary advantages of
classical 3D reconstruction and NeRF, we thus investigate marrying explicit
geometric representation with NeRF rendering to achieve efficient online
reconstruction and high-quality rendering. We introduce SurfelNeRF, a variant
of neural radiance field which employs a flexible and scalable neural surfel
representation to store geometric attributes and extracted appearance features
from input images. We further extend the conventional surfel-based fusion
scheme to progressively integrate incoming input frames into the reconstructed
global neural scene representation. In addition, we propose a highly-efficient
differentiable rasterization scheme for rendering neural surfel radiance
fields, which helps SurfelNeRF achieve $10\times$ speedups in training and
inference time, respectively. Experimental results show that our method
achieves the state-of-the-art 23.82 PSNR and 29.58 PSNR on ScanNet in
feedforward inference and per-scene optimization settings, respectively.

本文提出了 SurfelNeRF，这是一种基于神经辐射场的新型算法，将灵活可扩展的神经 Surfel 表示与 NeRF 渲染结合起来，用于高效的在线重建和高质量渲染大规模室内场景，并通过高效的可微栅格化方案实现了 10 倍的速度提升，并在 ScanNet 数据集上实现了最先进的结果。

SurfelNeRF: 神经 Surfel 辐射场用于在线照片般逼真的室内场景重建

SurfelNeRF: Neural Surfel Radiance Fields for Online Photorealistic  Reconstruction of Indoor Scenes

Creating 3D maps on robots and other mobile devices has become a reality in
recent years. Online 3D reconstruction enables many exciting applications in
robotics and AR/VR gaming. However, the reconstructions are noisy and generally
incomplete. Moreover, during onine reconstruction, the surface changes with
every newly integrated depth image which poses a significant challenge for
physics engines and path planning algorithms. This paper presents a novel, fast
and robust method for obtaining and using information about planar surfaces,
such as walls, floors, and ceilings as a stage in 3D reconstruction based on
Signed Distance Fields. Our algorithm recovers clean and accurate surfaces,
reduces the movement of individual mesh vertices caused by noise during online
reconstruction and fills in the occluded and unobserved regions. We implemented
and evaluated two different strategies to generate plane candidates and two
strategies for merging them. Our implementation is optimized to run in
real-time on mobile devices such as the Tango tablet. In an extensive set of
experiments, we validated that our approach works well in a large number of
natural environments despite the presence of significant amount of occlusion,
clutter and noise, which occur frequently. We further show that plane fitting
enables in many cases a meaningful semantic segmentation of real-world scenes.

本文提出一种通过使用有符号距离场获取和使用平面表面信息的新算法，有效地解决了在线重建中嘈杂不完整的问题，并使用平面拟合实现了意义上的语义分割。