Indoor scenes typically exhibit complex, spatially-varying appearance from
global illumination, making inverse rendering a challenging ill-posed problem.
This work presents an end-to-end, learning-based inverse rendering framework
incorporating differentiable Monte Carlo raytracing with importance sampling.
The framework takes a single image as input to jointly recover the underlying
geometry, spatially-varying lighting, and photorealistic materials.
Specifically, we introduce a physically-based differentiable rendering layer
with screen-space ray tracing, resulting in more realistic specular reflections
that match the input photo. In addition, we create a large-scale,
photorealistic indoor scene dataset with significantly richer details like
complex furniture and dedicated decorations. Further, we design a novel
out-of-view lighting network with uncertainty-aware refinement leveraging
hypernetwork-based neural radiance fields to predict lighting outside the view
of the input photo. Through extensive evaluations on common benchmark datasets,
we demonstrate superior inverse rendering quality of our method compared to
state-of-the-art baselines, enabling various applications such as complex
object insertion and material editing with high fidelity. Code and data will be
made available at https://jingsenzhu.github.io/invrend.

该研究提出一种终端到端的学习逆渲染框架，利用可微分 Monte Carlo 光线追踪结合重要性采样，从单张图像中恢复基础几何、空间变化的光照、真实感材料。通过物理上基于不同 iable 渲染层与荧幕空间光线追踪的引入，创建大规模室内场景数据集，设计一种新颖的场外灯光网络进行评估，呈现出与业界最先进基线方法相比提高了的反向渲染质量，可以实现诸如高度保真的复杂物体插入和材料编辑等各种应用。