In this work, we present a new multi-view depth estimation method that
utilizes both conventional reconstruction and learning-based priors over the
recently proposed neural radiance fields (NeRF). Unlike existing neural network
based optimization method that relies on estimated correspondences, our method
directly optimizes over implicit volumes, eliminating the challenging step of
matching pixels in indoor scenes. The key to our approach is to utilize the
learning-based priors to guide the optimization process of NeRF. Our system
firstly adapts a monocular depth network over the target scene by finetuning on
its sparse SfM+MVS reconstruction from COLMAP. Then, we show that the
shape-radiance ambiguity of NeRF still exists in indoor environments and
propose to address the issue by employing the adapted depth priors to monitor
the sampling process of volume rendering. Finally, a per-pixel confidence map
acquired by error computation on the rendered image can be used to further
improve the depth quality. Experiments show that our proposed framework
significantly outperforms state-of-the-art methods on indoor scenes, with
surprising findings presented on the effectiveness of correspondence-based
optimization and NeRF-based optimization over the adapted depth priors. In
addition, we show that the guided optimization scheme does not sacrifice the
original synthesis capability of neural radiance fields, improving the
rendering quality on both seen and novel views. Code is available at
this https URL

本文提出一种新的多视角深度估计方法，利用传统重建和基于学习的 prior，结合神经辐射场（NeRF）来直接优化隐式体积，消除室内场景中匹配像素的困难步骤，并使用自适应的深度先验来引导 NeRF 的优化过程，通过渲染图像计算出的逐像素置信度图可用于进一步提高深度质量。