Based on powerful text-to-image diffusion models, text-to-3D generation has made significant progress in generating compelling geometry and appearance. However, existing methods still struggle to recover high-fidelity object materials, either only considering Lambertian reflectance, or failing to disentangle BRDF materials from the environment lights. In this work, we propose Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR (\textbf{MATLABER}) that leverages a novel latent BRDF auto-encoder for material generation. We train this auto-encoder with large-scale real-world BRDF collections and ensure the smoothness of its latent space, which implicitly acts as a natural distribution of materials. During appearance modeling in text-to-3D generation, the latent BRDF embeddings, rather than BRDF parameters, are predicted via a material network. Through exhaustive experiments, our approach demonstrates the superiority over existing ones in generating realistic and coherent object materials. Moreover, high-quality materials naturally enable multiple downstream tasks such as relighting and material editing. Code and model will be publicly available at \url{https://sheldontsui.github.io/projects/Matlaber}.

基于强大的文本到图像扩散模型，文本到三维生成在生成引人注目的几何和外观方面取得了显著进展。本文提出了一种通过潜在BRDF自编码器生成材料的材料感知文本到三维生成方法（MATLABER）。通过在大规模真实世界的BRDF集合上训练这个自编码器，确保其潜在空间的平滑性，该空间在物料方面具有自然分布特性。在文本到三维生成中的外观建模过程中，通过材料网络预测潜在BRDF嵌入，而不是BRDF参数。通过详尽的实验，我们的方法在生成逼真连贯的物体材料方面表现出优越性。此外，高质量的材料自然地实现了多个下游任务，如重新照明和材料编辑。代码和模型将公开发布于https://sheldontsui.github.io/projects/Matlaber。

MATLABER: 材料感知性文本到三维的隐含BRDF自动编码器