Authoring high-quality digital materials is key to realism in 3D rendering.
Previous generative models for materials have been trained exclusively on
synthetic data; such data is limited in availability and has a visual gap to
real materials. We circumvent this limitation by proposing PhotoMat: the first
material generator trained exclusively on real photos of material samples
captured using a cell phone camera with flash. Supervision on individual
material maps is not available in this setting. Instead, we train a generator
for a neural material representation that is rendered with a learned relighting
module to create arbitrarily lit RGB images; these are compared against real
photos using a discriminator. We then train a material maps estimator to decode
material reflectance properties from the neural material representation. We
train PhotoMat with a new dataset of 12,000 material photos captured with
handheld phone cameras under flash lighting. We demonstrate that our generated
materials have better visual quality than previous material generators trained
on synthetic data. Moreover, we can fit analytical material models to closely
match these generated neural materials, thus allowing for further editing and
use in 3D rendering.

本研究提出了一种基于真实材料照片的首个材料生成器 PhotoMat，通过训练一个神经材料表征的生成器来创建任意光照的材料，从而弥补了不可用的合成数据，使得生成的材料在视觉上更加真实和精细，从而实现更加高质量的 3D 渲染。