In this work, we address the problem of jointly estimating albedo, normals,
depth and 3D spatially-varying lighting from a single image. Most existing
methods formulate the task as image-to-image translation, ignoring the 3D
properties of the scene. However, indoor scenes contain complex 3D light
transport where a 2D representation is insufficient. In this paper, we propose
a unified, learning-based inverse rendering framework that formulates 3D
spatially-varying lighting. Inspired by classic volume rendering techniques, we
propose a novel Volumetric Spherical Gaussian representation for lighting,
which parameterizes the exitant radiance of the 3D scene surfaces on a voxel
grid. We design a physics based differentiable renderer that utilizes our 3D
lighting representation, and formulates the energy-conserving image formation
process that enables joint training of all intrinsic properties with the
re-rendering constraint. Our model ensures physically correct predictions and
avoids the need for ground-truth HDR lighting which is not easily accessible.
Experiments show that our method outperforms prior works both quantitatively
and qualitatively, and is capable of producing photorealistic results for AR
applications such as virtual object insertion even for highly specular objects.

本文提出了一种用于室内场景的统一、基于学习的逆渲染框架，包括反照率、法线、深度和三维空间变化的照明的联合估计，其中设计了一种基于物理学的可微渲染器，利用我们的三维照明表示，形成能量守恒的图像形成过程，实现所有内在属性的联合训练，以重新渲染约束保证物理正确性，实验表明，该方法在量化和质量上优于之前的工作，能够为增强现实应用（例如虚拟物体插入）产生逼真的结果，即使对于高度镜面的物体也是如此。