We present AvatarPopUp, a method for fast, high quality 3D human avatar
generation from different input modalities, such as images and text prompts and
with control over the generated pose and shape. The common theme is the use of
diffusion-based image generation networks that are specialized for each
particular task, followed by a 3D lifting network. We purposefully decouple the
generation from the 3D modeling which allow us to leverage powerful image
synthesis priors, trained on billions of text-image pairs. We fine-tune latent
diffusion networks with additional image conditioning to solve tasks such as
image generation and back-view prediction, and to support qualitatively
different multiple 3D hypotheses. Our partial fine-tuning approach allows to
adapt the networks for each task without inducing catastrophic forgetting. In
our experiments, we demonstrate that our method produces accurate, high-quality
3D avatars with diverse appearance that respect the multimodal text, image, and
body control signals. Our approach can produce a 3D model in as few as 2
seconds, a four orders of magnitude speedup w.r.t. the vast majority of
existing methods, most of which solve only a subset of our tasks, and with
fewer controls, thus enabling applications that require the controlled 3D
generation of human avatars at scale. The project website can be found at
this https URL

通过 AvatarPopUp 方法，可以从不同输入模态生成快速且高质量的 3D 人体化身，具有对生成的姿势和形状的控制，使用基于扩散的图片生成网络和 3D 提升网络。该方法根据每个特定任务进行精细调优，能够在几秒内生成高质量的 3D 模型，远快于现有大多数方法，可以用于需要大规模控制 3D 生成的人体化身应用。