We introduce Reflectance Diffusion, a new neural text-to-texture model
capable of generating high-fidelity SVBRDF maps from textual descriptions. Our
method leverages a tandem neural approach, consisting of two modules, to
accurately model the distribution of spatially varying reflectance as described
by text prompts. Initially, we employ a pre-trained stable diffusion 2 model to
generate a latent representation that informs the overall shape of the material
and serves as our backbone model. Then, our ReflectanceUNet enables fine-tuning
control over the material's physical appearance and generates SVBRDF maps.
ReflectanceUNet module is trained on an extensive dataset comprising
approximately 200,000 synthetic spatially varying materials. Our generative
SVBRDF diffusion model allows for the synthesis of multiple SVBRDF estimates
from a single textual input, offering users the possibility to choose the
output that best aligns with their requirements. We illustrate our method's
versatility by generating SVBRDF maps from a range of textual descriptions,
both specific and broad. Our ReflectanceUNet model can integrate optional
physical parameters, such as roughness and specularity, enhancing
customization. When the backbone module is fixed, the ReflectanceUNet module
refines the material, allowing direct edits to its physical attributes.
Comparative evaluations demonstrate that ReflectanceFusion achieves better
accuracy than existing text-to-material models, such as Text2Mat, while also
providing the benefits of editable and relightable SVBRDF maps.

Reflectance Diffusion 是一种新的神经文本到纹理模型，从文本描述中生成高保真度的 SVBRDF 贴图。该方法通过两个模块的级联神经网络方法来准确地建模空间变化反射的分布。

反射融合：基于扩散的文本到 SVBRDF 生成

ReflectanceFusion: Diffusion-based text to SVBRDF Generation

Creating high quality and realistic materials in computer graphics is a
challenging and time-consuming task, which requires great expertise. In this
paper, we present MatFuse, a novel unified approach that harnesses the
generative power of diffusion models (DM) to simplify the creation of SVBRDF
maps. Our DM-based pipeline integrates multiple sources of conditioning, such
as color palettes, sketches, and pictures, enabling fine-grained control and
flexibility in material synthesis. This design allows for the combination of
diverse information sources (e.g., sketch + image embedding), enhancing
creative possibilities in line with the principle of compositionality. We
demonstrate the generative capabilities of the proposed method under various
conditioning settings; on the SVBRDF estimation task, we show that our method
yields performance comparable to state-of-the-art approaches, both
qualitatively and quantitatively.

MatFuse 是一种基于扩散模型的新型统一方法，用于简化计算机图形学中高质量逼真材料的创建，通过集成多种条件来源实现细粒度控制和材料合成的灵活性，该方法展示出与最先进方法相当的性能，同时丰富了创作可能性。