Learning neural implicit representations has achieved remarkable performance
in 3D reconstruction from multi-view images. Current methods use volume
rendering to render implicit representations into either RGB or depth images
that are supervised by multi-view ground truth. However, rendering a view each
time suffers from incomplete depth at holes and unawareness of occluded
structures from the depth supervision, which severely affects the accuracy of
geometry inference via volume rendering. To resolve this issue, we propose to
learn neural implicit representations from multi-view RGBD images through
volume rendering with an attentive depth fusion prior. Our prior allows neural
networks to perceive coarse 3D structures from the Truncated Signed Distance
Function (TSDF) fused from all depth images available for rendering. The TSDF
enables accessing the missing depth at holes on one depth image and the
occluded parts that are invisible from the current view. By introducing a novel
attention mechanism, we allow neural networks to directly use the depth fusion
prior with the inferred occupancy as the learned implicit function. Our
attention mechanism works with either a one-time fused TSDF that represents a
whole scene or an incrementally fused TSDF that represents a partial scene in
the context of Simultaneous Localization and Mapping (SLAM). Our evaluations on
widely used benchmarks including synthetic and real-world scans show our
superiority over the latest neural implicit methods. Project page:
this https URL

通过体积渲染和专注深度融合先验进行多视角 RGBD 图像学习神经隐式表示，以准确进行 3D 重建。该方法利用截断有符号距离函数（TSDF）从所有可用的深度图像融合并感知粗糙三维结构，解决了通过体积渲染进行几何推断的不完整深度和被遮挡结构的问题。通过引入注意机制，直接将深度融合先验与学习到的占用情况作为神经隐式函数，该机制可以用于整个场景或同时定位和映射（SLAM）背景下的部分场景。在合成和真实世界的广泛基准测试中，该方法超越了最新的神经隐式方法。