Material reconstruction from a photograph is a key component of 3D content
creation democratization. We propose to formulate this ill-posed problem as a
controlled synthesis one, leveraging the recent progress in generative deep
networks. We present ControlMat, a method which, given a single photograph with
uncontrolled illumination as input, conditions a diffusion model to generate
plausible, tileable, high-resolution physically-based digital materials. We
carefully analyze the behavior of diffusion models for multi-channel outputs,
adapt the sampling process to fuse multi-scale information and introduce rolled
diffusion to enable both tileability and patched diffusion for high-resolution
outputs. Our generative approach further permits exploration of a variety of
materials which could correspond to the input image, mitigating the unknown
lighting conditions. We show that our approach outperforms recent inference and
latent-space-optimization methods, and carefully validate our diffusion process
design choices. Supplemental materials and additional details are available at:
this https URL

通过生成深度网络和扩散模型，我们提出了一种方法来重建材料图像，使其生成逼真、平铺和高分辨率的基于物理的数码材料。