Large-scale pre-trained vision-language models allow for the zero-shot text-based generation of 3D avatars. The previous state-of-the-art method utilized CLIP to supervise neural implicit models that reconstructed a human body mesh. However, this approach has two limitations. Firstly, the lack of avatar-specific models can cause facial distortion and unrealistic clothing in the generated avatars. Secondly, CLIP only provides optimization direction for the overall appearance, resulting in less impressive results. To address these limitations, we propose AvatarFusion, the first framework to use a latent diffusion model to provide pixel-level guidance for generating human-realistic avatars while simultaneously segmenting clothing from the avatar's body. AvatarFusion includes the first clothing-decoupled neural implicit avatar model that employs a novel Dual Volume Rendering strategy to render the decoupled skin and clothing sub-models in one space. We also introduce a novel optimization method, called Pixel-Semantics Difference-Sampling (PS-DS), which semantically separates the generation of body and clothes, and generates a variety of clothing styles. Moreover, we establish the first benchmark for zero-shot text-to-avatar generation. Our experimental results demonstrate that our framework outperforms previous approaches, with significant improvements observed in all metrics. Additionally, since our model is clothing-decoupled, we can exchange the clothes of avatars. Code will be available on Github.

AvatarFusion利用一个潜在的扩散模型，同时从头到脚地生成人类逼真的头像，并将衣服与人物身体分割开来。AvatarFusion包括第一个脱离衣物的神经隐式头像模型，该模型采用了新的双体积渲染策略，将脱离衣物和衣物子模型在一个空间中渲染。我们还引入了一种新的优化方法，称为Pixel-Semantics Difference-Sampling（PS-DS），该方法在语义上将身体和衣物的生成分开，并生成各种服装风格。此外，由于我们的模型脱离了衣物，因此我们可以更换头像的衣服。

AvatarFusion: 2D扩散的零样本生成不受服装影响的3D化身