We present a deep learning approach to reconstruct scene appearance from unstructured images captured under collocated point lighting. At the heart of Deep Reflectance Volumes is a novel volumetric scene representation consisting of opacity, surface normal and reflectance voxel grids. We present a novel physically-based differentiable volume ray marching framework to render these scene volumes under arbitrary viewpoint and lighting. This allows us to optimize the scene volumes to minimize the error between their rendered images and the captured images. Our method is able to reconstruct real scenes with challenging non-Lambertian reflectance and complex geometry with occlusions and shadowing. Moreover, it accurately generalizes to novel viewpoints and lighting, including non-collocated lighting, rendering photorealistic images that are significantly better than state-of-the-art mesh-based methods. We also show that our learned reflectance volumes are editable, allowing for modifying the materials of the captured scenes.

本文提出了一种基于深度学习的场景重建方法，应用了一种新颖的体积场景表示方法——反射体积网格，通过物理可行的可微分体积光线行走框架，渲染出高质量的图像，实现了对具备挑战性的真实场景的重建，而重建得到的反射体积网格可进行编辑，可用于修改捕捉的场景材料。

多视角光度图像的深度反射成像：可重光重建