We present a novel method for 3D surface reconstruction from multiple images
where only a part of the object of interest is captured. Our approach builds on
two recent developments: surface reconstruction using neural radiance fields
for the reconstruction of the visible parts of the surface, and guidance of
pre-trained 2D diffusion models in the form of Score Distillation Sampling
(SDS) to complete the shape in unobserved regions in a plausible manner. We
introduce three components. First, we suggest employing normal maps as a pure
geometric representation for SDS instead of color renderings which are
entangled with the appearance information. Second, we introduce the freezing of
the SDS noise during training which results in more coherent gradients and
better convergence. Third, we propose Multi-View SDS as a way to condition the
generation of the non-observable part of the surface without fine-tuning or
making changes to the underlying 2D Stable Diffusion model. We evaluate our
approach on the BlendedMVS dataset demonstrating significant qualitative and
quantitative improvements over competing methods.

我们提出了一种新颖的方法，用于从多个图像中进行 3D 物体表面重建，其中只捕获了物体的一部分。我们的方法基于两个最新发展：使用神经辐射场进行表面重建，用于重建表面的可见部分，以及使用预训练的 2D 扩散模型的指导（SDS）完成未观察到的区域的形状。我们引入了三个组件：首先，建议使用法线图作为 SDS 的纯几何表示，而不是与外观信息交织在一起的颜色渲染。其次，我们在训练过程中使 SDS 噪声保持固定，这会导致更一致的梯度和更好的收敛性。第三，我们提出了多视角 SDS 作为一种在不对底层 2D 稳定扩散模型进行微调或更改的情况下对非可观察部分的生成进行条件化的方法。我们在 BlendedMVS 数据集上对我们的方法进行评估，结果显示相比竞争方法，我们取得了显著的定量和定性改进。