We concentrate on a novel human-centric image synthesis task, that is, given
only one reference facial photograph, it is expected to generate specific
individual images with diverse head positions, poses, and facial expressions in
different contexts. To accomplish this goal, we argue that our generative model
should be capable of the following favorable characteristics: (1) a strong
visual and semantic understanding of our world and human society for basic
object and human image generation. (2) generalizable identity preservation
ability. (3) flexible and fine-grained head control. Recently, large
pre-trained text-to-image diffusion models have shown remarkable results,
serving as a powerful generative foundation. As a basis, we aim to unleash the
above two capabilities of the pre-trained model. In this work, we present a new
framework named CapHuman. We embrace the ``encode then learn to align"
paradigm, which enables generalizable identity preservation for new individuals
without cumbersome tuning at inference. CapHuman encodes identity features and
then learns to align them into the latent space. Moreover, we introduce the 3D
facial prior to equip our model with control over the human head in a flexible
and 3D-consistent manner. Extensive qualitative and quantitative analyses
demonstrate our CapHuman can produce well-identity-preserved, photo-realistic,
and high-fidelity portraits with content-rich representations and various head
renditions, superior to established baselines. Code and checkpoint will be
released at this https URL

我们提出了一种名为 CapHuman 的新框架，通过 “编码然后学习对齐” 的范例，能够在推理过程中对新的个体进行可推广的身份保留，通过引入三维面部先验以实现对人头的灵活和三维一致的控制，从而产生具有丰富内容表示和各种不同头部造型、超越现有基线模型的身份保留、逼真和高保真肖像画。