Manually creating textures for 3D meshes is time-consuming, even for expert
visual content creators. We propose a fast approach for automatically texturing
an input 3D mesh based on a user-provided text prompt. Importantly, our
approach disentangles lighting from surface material/reflectance in the
resulting texture so that the mesh can be properly relit and rendered in any
lighting environment. We introduce LightControlNet, a new text-to-image model
based on the ControlNet architecture, which allows the specification of the
desired lighting as a conditioning image to the model. Our text-to-texture
pipeline then constructs the texture in two stages. The first stage produces a
sparse set of visually consistent reference views of the mesh using
LightControlNet. The second stage applies a texture optimization based on Score
Distillation Sampling (SDS) that works with LightControlNet to increase the
texture quality while disentangling surface material from lighting. Our
pipeline is significantly faster than previous text-to-texture methods, while
producing high-quality and relightable textures.

基于用户提供的文本提示，我们提出了一种快速的自动给输入的 3D 网格上色的方法。我们的方法将光照与表面材质 / 反射分离，以便在任何光照环境中正确照明和渲染网格。我们介绍了一种基于 ControlNet 架构的新的文本到图像模型 LightControlNet，它允许将期望的光照规范为模型的一个条件图像。然后，我们的文本到纹理流程通过两个阶段构建纹理。第一阶段使用 LightControlNet 产生一组稀疏的视觉一致的网格参考视图。第二阶段根据得分蒸馏采样 (SDS) 进行纹理优化，该优化方法与 LightControlNet 配合工作，以提高纹理质量并分离表面材质和光照。我们的流程比先前的文本到纹理方法快得多，同时能产生高质量且可重照的纹理。