DiffusionAvatars synthesizes a high-fidelity 3D head avatar of a person,
offering intuitive control over both pose and expression. We propose a
diffusion-based neural renderer that leverages generic 2D priors to produce
compelling images of faces. For coarse guidance of the expression and head
pose, we render a neural parametric head model (NPHM) from the target
viewpoint, which acts as a proxy geometry of the person. Additionally, to
enhance the modeling of intricate facial expressions, we condition
DiffusionAvatars directly on the expression codes obtained from NPHM via
cross-attention. Finally, to synthesize consistent surface details across
different viewpoints and expressions, we rig learnable spatial features to the
head's surface via TriPlane lookup in NPHM's canonical space. We train
DiffusionAvatars on RGB videos and corresponding tracked NPHM meshes of a
person and test the obtained avatars in both self-reenactment and animation
scenarios. Our experiments demonstrate that DiffusionAvatars generates
temporally consistent and visually appealing videos for novel poses and
expressions of a person, outperforming existing approaches.

DiffusionAvatars 通过合成高保真度的三维头像人物，提供了对姿态和表情的直观控制。我们提出了一种基于扩散的神经渲染器，利用通用的二维先验生成引人注目的人脸图像。通过从目标视点渲染神经参数头模型（NPHM）对表情和头部姿态进行粗略指导，我们作为人物的代理几何体。此外，为了增强精细面部表情的建模，我们直接在 NPHM 中通过交叉注意力采用从 NPHM 获得的表情代码来调整 DiffusionAvatars。最后，为了在不同视点和表情之间综合一致地细化表面细节，我们通过 NPHM 的规范空间在头部表面上装配可学习的空间特征。我们使用人物的 RGB 视频和相应跟踪的 NPHM 网格对 DiffusionAvatars 进行训练，并在自我重演和动画场景中测试所得到的头像。我们的实验证明，DiffusionAvatars 在生成人物的新姿态和表情的时候能够产生时间上连贯而视觉上有吸引力的视频，优于现有方法。

DiffusionAvatars：高保真 3D 头像的延迟扩散

DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars

We present a method that reconstructs and animates a 3D head avatar from a
single-view portrait image. Existing methods either involve time-consuming
optimization for a specific person with multiple images, or they struggle to
synthesize intricate appearance details beyond the facial region. To address
these limitations, we propose a framework that not only generalizes to unseen
identities based on a single-view image without requiring person-specific
optimization, but also captures characteristic details within and beyond the
face area (e.g. hairstyle, accessories, etc.). At the core of our method are
three branches that produce three tri-planes representing the coarse 3D
geometry, detailed appearance of a source image, as well as the expression of a
target image. By applying volumetric rendering to the combination of the three
tri-planes followed by a super-resolution module, our method yields a high
fidelity image of the desired identity, expression and pose. Once trained, our
model enables efficient 3D head avatar reconstruction and animation via a
single forward pass through a network. Experiments show that the proposed
approach generalizes well to unseen validation datasets, surpassing SOTA
baseline methods by a large margin on head avatar reconstruction and animation.

该研究提出了一种通过对单视角幅面图像进行重建和动画化的三维头像，通过三个三角面分别捕捉源图像的粗略三维几何，详细外观以及目标图像的表情实现，再通过超分辨率模块的渲染进行填充并通过单向传递网络的高效性进行动画。实验表明，该方法在重建和动画方面优于当前最先进的基线方法，并且对新的验证数据集有很好的泛化能力。