In this study, we propose AniPortrait, a novel framework for generating
high-quality animation driven by audio and a reference portrait image. Our
methodology is divided into two stages. Initially, we extract 3D intermediate
representations from audio and project them into a sequence of 2D facial
landmarks. Subsequently, we employ a robust diffusion model, coupled with a
motion module, to convert the landmark sequence into photorealistic and
temporally consistent portrait animation. Experimental results demonstrate the
superiority of AniPortrait in terms of facial naturalness, pose diversity, and
visual quality, thereby offering an enhanced perceptual experience. Moreover,
our methodology exhibits considerable potential in terms of flexibility and
controllability, which can be effectively applied in areas such as facial
motion editing or face reenactment. We release code and model weights at
this https URL

提出了 AniPortrait，这是一个由音频和参考肖像图像驱动的，用于生成高质量动画的新框架。通过从音频提取 3D 中间表示并将其投影到一系列 2D 面部标志中，然后使用强大的扩散模型和动画模块将标志序列转换为逼真且时间连续的肖像动画，实验结果表明 AniPortrait 在面部自然度、姿势多样性和视觉质量方面具有优势，因此提供了增强的感知体验。此外，我们的方法在灵活性和可控性方面具有相当大的潜力，可在面部动作编辑或面部再现等领域有效应用。