We introduce Reflectance Diffusion, a new neural text-to-texture model
capable of generating high-fidelity SVBRDF maps from textual descriptions. Our
method leverages a tandem neural approach, consisting of two modules, to
accurately model the distribution of spatially varying reflectance as described
by text prompts. Initially, we employ a pre-trained stable diffusion 2 model to
generate a latent representation that informs the overall shape of the material
and serves as our backbone model. Then, our ReflectanceUNet enables fine-tuning
control over the material's physical appearance and generates SVBRDF maps.
ReflectanceUNet module is trained on an extensive dataset comprising
approximately 200,000 synthetic spatially varying materials. Our generative
SVBRDF diffusion model allows for the synthesis of multiple SVBRDF estimates
from a single textual input, offering users the possibility to choose the
output that best aligns with their requirements. We illustrate our method's
versatility by generating SVBRDF maps from a range of textual descriptions,
both specific and broad. Our ReflectanceUNet model can integrate optional
physical parameters, such as roughness and specularity, enhancing
customization. When the backbone module is fixed, the ReflectanceUNet module
refines the material, allowing direct edits to its physical attributes.
Comparative evaluations demonstrate that ReflectanceFusion achieves better
accuracy than existing text-to-material models, such as Text2Mat, while also
providing the benefits of editable and relightable SVBRDF maps.

Reflectance Diffusion 是一种新的神经文本到纹理模型，从文本描述中生成高保真度的 SVBRDF 贴图。该方法通过两个模块的级联神经网络方法来准确地建模空间变化反射的分布。