Synthesizing photorealistic 4D human head avatars from videos is essential
for VR/AR, telepresence, and video game applications. Although existing Neural
Radiance Fields (NeRF)-based methods achieve high-fidelity results, the
computational expense limits their use in real-time applications. To overcome
this limitation, we introduce BakedAvatar, a novel representation for real-time
neural head avatar synthesis, deployable in a standard polygon rasterization
pipeline. Our approach extracts deformable multi-layer meshes from learned
isosurfaces of the head and computes expression-, pose-, and view-dependent
appearances that can be baked into static textures for efficient rasterization.
We thus propose a three-stage pipeline for neural head avatar synthesis, which
includes learning continuous deformation, manifold, and radiance fields,
extracting layered meshes and textures, and fine-tuning texture details with
differential rasterization. Experimental results demonstrate that our
representation generates synthesis results of comparable quality to other
state-of-the-art methods while significantly reducing the inference time
required. We further showcase various head avatar synthesis results from
monocular videos, including view synthesis, face reenactment, expression
editing, and pose editing, all at interactive frame rates.

实时神经头像合成方法 BakedAvatar 可用于 VR/AR、远程呈现和视频游戏应用中，通过提取多层可变形网格和计算表情、姿势和视角相关的外观，将结果烘培为静态贴图以提高栅格化效率，并通过差分栅格化来优化贴图细节，从而在减少推理时间的同时生成与先进方法相当质量的合成结果。