3D head avatars built with neural implicit volumetric representations have
achieved unprecedented levels of photorealism. However, the computational cost
of these methods remains a significant barrier to their widespread adoption,
particularly in real-time applications such as virtual reality and
teleconferencing. While attempts have been made to develop fast neural
rendering approaches for static scenes, these methods cannot be simply employed
to support realistic facial expressions, such as in the case of a dynamic
facial performance. To address these challenges, we propose a novel fast 3D
neural implicit head avatar model that achieves real-time rendering while
maintaining fine-grained controllability and high rendering quality. Our key
idea lies in the introduction of local hash table blendshapes, which are
learned and attached to the vertices of an underlying face parametric model.
These per-vertex hash-tables are linearly merged with weights predicted via a
CNN, resulting in expression dependent embeddings. Our novel representation
enables efficient density and color predictions using a lightweight MLP, which
is further accelerated by a hierarchical nearest neighbor search method.
Extensive experiments show that our approach runs in real-time while achieving
comparable rendering quality to state-of-the-arts and decent results on
challenging expressions.

提出了一种新颖的快速 3D 神经隐式头像模型，实现实时渲染并保持细粒度的可控性和高渲染质量。本方法引入局部哈希表混合形状，通过线性合并和卷积神经网络预测的权重，将其附加到底层面部参数模型的顶点上，从而实现表情相关的嵌入。通过轻量级多层感知机实现高效的密度和颜色预测，并通过分层最近邻搜索方法加速。大量实验证明，本方法在实时运行的同时实现了与最新技术相比可比的渲染质量，并对具有挑战性的表情获得了不错的结果。

具有网格锚定哈希表融合形状的高效三维隐式头像

Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table  Blendshapes

Recently, text-guided 3D generative methods have made remarkable advancements
in producing high-quality textures and geometry, capitalizing on the
proliferation of large vision-language and image diffusion models. However,
existing methods still struggle to create high-fidelity 3D head avatars in two
aspects: (1) They rely mostly on a pre-trained text-to-image diffusion model
whilst missing the necessary 3D awareness and head priors. This makes them
prone to inconsistency and geometric distortions in the generated avatars. (2)
They fall short in fine-grained editing. This is primarily due to the inherited
limitations from the pre-trained 2D image diffusion models, which become more
pronounced when it comes to 3D head avatars. In this work, we address these
challenges by introducing a versatile coarse-to-fine pipeline dubbed HeadSculpt
for crafting (i.e., generating and editing) 3D head avatars from textual
prompts. Specifically, we first equip the diffusion model with 3D awareness by
leveraging landmark-based control and a learned textual embedding representing
the back view appearance of heads, enabling 3D-consistent head avatar
generations. We further propose a novel identity-aware editing score
distillation strategy to optimize a textured mesh with a high-resolution
differentiable rendering technique. This enables identity preservation while
following the editing instruction. We showcase HeadSculpt's superior fidelity
and editing capabilities through comprehensive experiments and comparisons with
existing methods.

本研究提出了名为 HeadSculpt 的通用粗到细流程，用于从文本提示中生成和编辑三维头像，实现了更高的保真度和编辑能力。

HeadSculpt: 使用文本创造 3D 头像

HeadSculpt: Crafting 3D Head Avatars with Text

High-quality reconstruction of controllable 3D head avatars from 2D videos is
highly desirable for virtual human applications in movies, games, and
telepresence. Neural implicit fields provide a powerful representation to model
3D head avatars with personalized shape, expressions, and facial parts, e.g.,
hair and mouth interior, that go beyond the linear 3D morphable model (3DMM).
However, existing methods do not model faces with fine-scale facial features,
or local control of facial parts that extrapolate asymmetric expressions from
monocular videos. Further, most condition only on 3DMM parameters with poor(er)
locality, and resolve local features with a global neural field. We build on
part-based implicit shape models that decompose a global deformation field into
local ones. Our novel formulation models multiple implicit deformation fields
with local semantic rig-like control via 3DMM-based parameters, and
representative facial landmarks. Further, we propose a local control loss and
attention mask mechanism that promote sparsity of each learned deformation
field. Our formulation renders sharper locally controllable nonlinear
deformations than previous implicit monocular approaches, especially mouth
interior, asymmetric expressions, and facial details.

通过基于部件的隐式形状模型，我们设计了一种可以控制 3D 头像的方法，可以推断局部语义 Rig，包括面部特征，通过局部控制丰富了隐式形变字段，使其具有更强的局部控制性能。