Video-driven neural face reenactment aims to synthesize realistic facial
images that successfully preserve the identity and appearance of a source face,
while transferring the target head pose and facial expressions. Existing
GAN-based methods suffer from either distortions and visual artifacts or poor
reconstruction quality, i.e., the background and several important appearance
details, such as hair style/color, glasses and accessories, are not faithfully
reconstructed. Recent advances in Diffusion Probabilistic Models (DPMs) enable
the generation of high-quality realistic images. To this end, in this paper we
present DiffusionAct, a novel method that leverages the photo-realistic image
generation of diffusion models to perform neural face reenactment.
Specifically, we propose to control the semantic space of a Diffusion
Autoencoder (DiffAE), in order to edit the facial pose of the input images,
defined as the head pose orientation and the facial expressions. Our method
allows one-shot, self, and cross-subject reenactment, without requiring
subject-specific fine-tuning. We compare against state-of-the-art GAN-,
StyleGAN2-, and diffusion-based methods, showing better or on-par reenactment
performance.

DiffusionAct 是一种利用扩散模型的照片级真实图像生成来进行神经脸部再现的新方法，能够编辑输入图像的脸部姿势，实现一次性、自我和跨主体再现，并展示了优于甚至与 GAN、StyleGAN2 和扩散方法相媲美的再现性能。