We address the problem of reconstructing spatially-varying BRDFs from a small
set of image measurements. This is a fundamentally under-constrained problem,
and previous work has relied on using various regularization priors or on
capturing many images to produce plausible results. In this work, we present
MaterialGAN, a deep generative convolutional network based on StyleGAN2,
trained to synthesize realistic SVBRDF parameter maps. We show that MaterialGAN
can be used as a powerful material prior in an inverse rendering framework: we
optimize in its latent representation to generate material maps that match the
appearance of the captured images when rendered. We demonstrate this framework
on the task of reconstructing SVBRDFs from images captured under flash
illumination using a hand-held mobile phone. Our method succeeds in producing
plausible material maps that accurately reproduce the target images, and
outperforms previous state-of-the-art material capture methods in evaluations
on both synthetic and real data. Furthermore, our GAN-based latent space allows
for high-level semantic material editing operations such as generating material
variations and material morphing.

本文利用基于 StyleGAN2 的深度生成卷积网络 MaterialGAN，通过在其潜在表示空间中进行优化，将闪光灯照射下拍摄的图像转化成能够精确还原目标图像的材料贴图，并取得了优于现有技术水平的结果。此外，我们还展示了 MaterialGAN 的高级语义材料编辑操作，例如材料变化和材料形态变化。