We present a differentiable rendering framework for material and lighting
estimation from multi-view images and a reconstructed geometry. In the
framework, we represent scene lightings as the Neural Incident Light Field
(NeILF) and material properties as the surface BRDF modelled by multi-layer
perceptrons. Compared with recent approaches that approximate scene lightings
as the 2D environment map, NeILF is a fully 5D light field that is capable of
modelling illuminations of any static scenes. In addition, occlusions and
indirect lights can be handled naturally by the NeILF representation without
requiring multiple bounces of ray tracing, making it possible to estimate
material properties even for scenes with complex lightings and geometries. We
also propose a smoothness regularization and a Lambertian assumption to reduce
the material-lighting ambiguity during the optimization. Our method strictly
follows the physically-based rendering equation, and jointly optimizes material
and lighting through the differentiable rendering process. We have intensively
evaluated the proposed method on our in-house synthetic dataset, the DTU MVS
dataset, and real-world BlendedMVS scenes. Our method is able to outperform
previous methods by a significant margin in terms of novel view rendering
quality, setting a new state-of-the-art for image-based material and lighting
estimation.

本文提出了基于可微分渲染的方法，通过神经入射光场和多层感知机模擬的表面 BRDF 模型来实现多视图图像和重构几何体的物质和照明估计，该方法能够准确估计静态场景的照明，并可自然处理遮挡和间接光线，未需多次反射。我们在多个场景数据集上实验，结果表明本方法以显著的优势超越现有方法，成为图像物质和照明估计的新的最佳实践。

NeILF: 基于神经网络的物理感材料估计的入射光场

NeILF: Neural Incident Light Field for Physically-based Material  Estimation

We introduce a novel neural network-based BRDF model and a Bayesian framework
for object inverse rendering, i.e., joint estimation of reflectance and natural
illumination from a single image of an object of known geometry. The BRDF is
expressed with an invertible neural network, namely, normalizing flow, which
provides the expressive power of a high-dimensional representation,
computational simplicity of a compact analytical model, and physical
plausibility of a real-world BRDF. We extract the latent space of real-world
reflectance by conditioning this model, which directly results in a strong
reflectance prior. We refer to this model as the invertible neural BRDF model
(iBRDF). We also devise a deep illumination prior by leveraging the structural
bias of deep neural networks. By integrating this novel BRDF model and
reflectance and illumination priors in a MAP estimation formulation, we show
that this joint estimation can be computed efficiently with stochastic gradient
descent. We experimentally validate the accuracy of the invertible neural BRDF
model on a large number of measured data and demonstrate its use in object
inverse rendering on a number of synthetic and real images. The results show
new ways in which deep neural networks can help solve challenging radiometric
inverse problems.

本文提出了一种新的神经网络基于 BRDF 模型和贝叶斯框架，用于从一个已知几何物体的单个图像中联合估计反射和自然照明。该模型用可逆神经网络表达反射率，利用高维表示的表达能力，紧凑分析模型的计算简洁性和真实世界 BRDF 的物理可行性。实验表明，这种神经网络可帮助解决具有挑战性的辐射反问题。