In this paper, we present GaNI, a Global and Near-field Illumination-aware
neural inverse rendering technique that can reconstruct geometry, albedo, and
roughness parameters from images of a scene captured with co-located light and
camera. Existing inverse rendering techniques with co-located light-camera
focus on single objects only, without modeling global illumination and
near-field lighting more prominent in scenes with multiple objects. We
introduce a system that solves this problem in two stages; we first reconstruct
the geometry powered by neural volumetric rendering NeuS, followed by inverse
neural radiosity that uses the previously predicted geometry to estimate albedo
and roughness. However, such a naive combination fails and we propose multiple
technical contributions that enable this two-stage approach. We observe that
NeuS fails to handle near-field illumination and strong specular reflections
from the flashlight in a scene. We propose to implicitly model the effects of
near-field illumination and introduce a surface angle loss function to handle
specular reflections. Similarly, we observe that invNeRad assumes constant
illumination throughout the capture and cannot handle moving flashlights during
capture. We propose a light position-aware radiance cache network and
additional smoothness priors on roughness to reconstruct reflectance.
Experimental evaluation on synthetic and real data shows that our method
outperforms the existing co-located light-camera-based inverse rendering
techniques. Our approach produces significantly better reflectance and slightly
better geometry than capture strategies that do not require a dark room.

本研究提出了一种名为 GaNI 的全球和近场照明感知的神经逆渲染技术，能够从光和相机共同拍摄的场景图像中重建几何、反射率和粗糙度参数。我们通过两个阶段的方法来解决现有的逆渲染技术只能针对单个物体的问题，首先利用神经体积渲染 NeuS 重建几何，然后使用先前预测的几何来估计反射率和粗糙度。实验证明，我们的方法在合成和真实数据上表现优于现有的基于光和相机共同拍摄的逆渲染技术，能够产生更好的反射率和略微更好的几何。