Multi-view inverse rendering is the problem of estimating the scene
parameters such as shapes, materials, or illuminations from a sequence of
images captured under different viewpoints. Many approaches, however, assume
single light bounce and thus fail to recover challenging scenarios like
inter-reflections. On the other hand, simply extending those methods to
consider multi-bounced light requires more assumptions to alleviate the
ambiguity. To address this problem, we propose Neural Incident Stokes Fields
(NeISF), a multi-view inverse rendering framework that reduces ambiguities
using polarization cues. The primary motivation for using polarization cues is
that it is the accumulation of multi-bounced light, providing rich information
about geometry and material. Based on this knowledge, the proposed incident
Stokes field efficiently models the accumulated polarization effect with the
aid of an original physically-based differentiable polarimetric renderer.
Lastly, experimental results show that our method outperforms the existing
works in synthetic and real scenarios.

从不同视角下的图像序列中估计场景参数的多视角逆渲染问题，通过利用极化线索来减少歧义性，以物理为基础的可微分极化渲染器辅助下，提出了神经入射斯托克斯场（NeISF）的多视角逆渲染框架。实验结果表明，该方法在合成和真实场景中优于现有作品。

NeISF：神经事故斯托克斯场的几何和材料估计

NeISF: Neural Incident Stokes Field for Geometry and Material Estimation

We present a efficient multi-view inverse rendering method for large-scale
real-world indoor scenes that reconstructs global illumination and
physically-reasonable SVBRDFs. Unlike previous representations, where the
global illumination of large scenes is simplified as multiple environment maps,
we propose a compact representation called Texture-based Lighting (TBL). It
consists of 3D mesh and HDR textures, and efficiently models direct and
infinite-bounce indirect lighting of the entire large scene. Based on TBL, we
further propose a hybrid lighting representation with precomputed irradiance,
which significantly improves the efficiency and alleviates the rendering noise
in the material optimization. To physically disentangle the ambiguity between
materials, we propose a three-stage material optimization strategy based on the
priors of semantic segmentation and room segmentation. Extensive experiments
show that the proposed method outperforms the state-of-the-art quantitatively
and qualitatively, and enables physically-reasonable mixed-reality applications
such as material editing, editable novel view synthesis and relighting. The
project page is at this https URL

本文提出了一种高效的多视角逆向呈现方法，可以重建大规模室内场景的全局间接光照和物理合理的 SVBRDF，通过所提出的 Texture-Based Lighting (TBL) 来实现整个大场景的直接光照和无限反弹间接光照的紧凑表示，并基于 TBL 提出了一种预计算辐照度的混合光照表示，通过基于语义分割和房间分割的先验提出了三阶段材料优化策略来消除材料之间的歧义，实验结果表明所提出的方法在定量和定性上均优于现有的最佳方法，并启用了物理合理的增强现实应用，如材质编辑，可编辑的新视角合成和重新照明。

大规模真实室内场景多视角反渲染

Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes

Reconstructing an object's high-quality 3D shape with inherent spectral
reflectance property, beyond typical device-dependent RGB albedos, opens the
door to applications requiring a high-fidelity 3D model in terms of both
geometry and photometry. In this paper, we propose a novel Multi-View Inverse
Rendering (MVIR) method called Spectral MVIR for jointly reconstructing the 3D
shape and the spectral reflectance for each point of object surfaces from
multi-view images captured using a standard RGB camera and low-cost lighting
equipment such as an LED bulb or an LED projector. Our main contributions are
twofold: (i) We present a rendering model that considers both geometric and
photometric principles in the image formation by explicitly considering camera
spectral sensitivity, light's spectral power distribution, and light source
positions. (ii) Based on the derived model, we build a cost-optimization MVIR
framework for the joint reconstruction of the 3D shape and the per-vertex
spectral reflectance while estimating the light source positions and the
shadows. Different from most existing spectral-3D acquisition methods, our
method does not require expensive special equipment and cumbersome geometric
calibration. Experimental results using both synthetic and real-world data
demonstrate that our Spectral MVIR can acquire a high-quality 3D model with
accurate spectral reflectance property.

本文提出了一种名为 Spectral MVIR 的多视角逆向渲染 (MVIR) 方法，可以使用标准 RGB 相机和低成本的照明设备，联合重构物体表面每个点的 3D 形状和光谱反射率。该方法不需要昂贵的特殊设备和繁琐的几何校准，实验结果表明，该方法能够获得具有准确光谱反射率性质的高质量 3D 模型。