We introduce an approach for 3D head avatar generation and editing with
multi-modal conditioning based on a 3D Generative Adversarial Network (GAN) and
a Latent Diffusion Model (LDM). 3D GANs can generate high-quality head avatars
given a single or no condition. However, it is challenging to generate samples
that adhere to multiple conditions of different modalities. On the other hand,
LDMs excel at learning complex conditional distributions. To this end, we
propose to exploit the conditioning capabilities of LDMs to enable multi-modal
control over the latent space of a pre-trained 3D GAN. Our method can generate
and edit 3D head avatars given a mixture of control signals such as RGB input,
segmentation masks, and global attributes. This provides better control over
the generation and editing of synthetic avatars both globally and locally.
Experiments show that our proposed approach outperforms a solely GAN-based
approach both qualitatively and quantitatively on generation and editing tasks.
To the best of our knowledge, our approach is the first to introduce
multi-modal conditioning to 3D avatar generation and editing.
\\href{avatarmmc-sig24.github.io}{Project Page}

我们介绍了一种基于 3D 生成对抗网络（GAN）和潜在扩散模型（LDM）的多模态条件的 3D 头像生成和编辑方法。我们的方法可以根据混合控制信号（例如 RGB 输入，分割掩模和全局属性）生成和编辑 3D 头像，从而在全局和局部方面提供对合成头像的更好控制。