Landmarks often play a key role in face analysis, but many aspects of
identity or expression cannot be represented by sparse landmarks alone. Thus,
in order to reconstruct faces more accurately, landmarks are often combined
with additional signals like depth images or techniques like differentiable
rendering. Can we keep things simple by just using more landmarks? In answer,
we present the first method that accurately predicts 10x as many landmarks as
usual, covering the whole head, including the eyes and teeth. This is
accomplished using synthetic training data, which guarantees perfect landmark
annotations. By fitting a morphable model to these dense landmarks, we achieve
state-of-the-art results for monocular 3D face reconstruction in the wild. We
show that dense landmarks are an ideal signal for integrating face shape
information across frames by demonstrating accurate and expressive facial
performance capture in both monocular and multi-view scenarios. This approach
is also highly efficient: we can predict dense landmarks and fit our 3D face
model at over 150FPS on a single CPU thread. Please see our website:
this https URL

该研究论文提出了一种通过使用合成数据达到高密度标记点的方法来进行面部重建的方法，并成功将其用于嘴巴和眼睛等部位进行表情捕捉，从而取得了在单目 3D 面部重建方面的最新成果。

稠密特征点的 3D 面部重建

3D face reconstruction with dense landmarks

We present Neural Head Avatars, a novel neural representation that explicitly
models the surface geometry and appearance of an animatable human avatar that
can be used for teleconferencing in AR/VR or other applications in the movie or
games industry that rely on a digital human. Our representation can be learned
from a monocular RGB portrait video that features a range of different
expressions and views. Specifically, we propose a hybrid representation
consisting of a morphable model for the coarse shape and expressions of the
face, and two feed-forward networks, predicting vertex offsets of the
underlying mesh as well as a view- and expression-dependent texture. We
demonstrate that this representation is able to accurately extrapolate to
unseen poses and view points, and generates natural expressions while providing
sharp texture details. Compared to previous works on head avatars, our method
provides a disentangled shape and appearance model of the complete human head
(including hair) that is compatible with the standard graphics pipeline.
Moreover, it quantitatively and qualitatively outperforms current state of the
art in terms of reconstruction quality and novel-view synthesis.

我们提出了神经头像，该方法能够显式建模动画人物头像的表面几何形状和外观，是虚拟现实和其他电影或游戏应用中数字人物的一种有效表示方法，它能够从单目 RGB 人像视频中学习，能够精确地外推到未知姿态和观察角度，并产生自然的表情和清晰的纹理细节。

单目 RGB 视频生成神经头像

Neural Head Avatars from Monocular RGB Videos

This paper presents a novel approach for synthesizing facial affect; either
in terms of the six basic expressions (i.e., anger, disgust, fear, joy, sadness
and surprise), or in terms of valence (i.e., how positive or negative is an
emotion) and arousal (i.e., power of the emotion activation). The proposed
approach accepts the following inputs: i) a neutral 2D image of a person; ii) a
basic facial expression or a pair of valence-arousal (VA) emotional state
descriptors to be generated, or a path of affect in the 2D VA Space to be
generated as an image sequence. In order to synthesize affect in terms of VA,
for this person, $600,000$ frames from the 4DFAB database were annotated. The
affect synthesis is implemented by fitting a 3D Morphable Model on the neutral
image, then deforming the reconstructed face and adding the inputted affect,
and blending the new face with the given affect into the original image.
Qualitative experiments illustrate the generation of realistic images, when the
neutral image is sampled from thirteen well known lab-controlled or in-the-wild
databases, including Aff-Wild, AffectNet, RAF-DB; comparisons with Generative
Adversarial Networks (GANs) show the higher quality achieved by the proposed
approach. Then, quantitative experiments are conducted, in which the
synthesized images are used for data augmentation in training Deep Neural
Networks to perform affect recognition over all databases; greatly improved
performances are achieved when compared with state-of-the-art methods, as well
as with GAN-based data augmentation, in all cases.

本文提出了一种新的方法，用于合成面部情感，接受中性 2D 人脸图像、基本面部表情或 VA 头像建议，以及 2D VA 空间中的路径作为输入，采用 3D 可变模型，对中性人脸图像进行形变和添加颜色，使生成的面孔带有特定情感，用于数据增强和深度神经网络的联调。