Digital humans and, especially, 3D facial avatars have raised a lot of
attention in the past years, as they are the backbone of several applications
like immersive telepresence in AR or VR. Despite the progress, facial avatars
reconstructed from commodity hardware are incomplete and miss out on parts of
the side and back of the head, severely limiting the usability of the avatar.
This limitation in prior work stems from their requirement of face tracking,
which fails for profile and back views. To address this issue, we propose to
learn person-specific animatable avatars from images without assuming to have
access to precise facial expression tracking. At the core of our method, we
leverage a 3D-aware generative model that is trained to reproduce the
distribution of facial expressions from the training data. To train this
appearance model, we only assume to have a collection of 2D images with the
corresponding camera parameters. For controlling the model, we learn a mapping
from 3DMM facial expression parameters to the latent space of the generative
model. This mapping can be learned by sampling the latent space of the
appearance model and reconstructing the facial parameters from a normalized
frontal view, where facial expression estimation performs well. With this
scheme, we decouple 3D appearance reconstruction and animation control to
achieve high fidelity in image synthesis. In a series of experiments, we
compare our proposed technique to state-of-the-art monocular methods and show
superior quality while not requiring expression tracking of the training data.

我们提出了一种从图像中学习具有人物特定的可动画化角色模型的方法，旨在解决面部表情追踪失败的问题，并实现高保真度的图像合成。

GAN-Avatar: 可控个性化基于 GAN 的人头图像生成器

GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar

Monocular 3D human performance capture is indispensable for many applications
in computer graphics and vision for enabling immersive experiences. However,
detailed capture of humans requires tracking of multiple aspects, including the
skeletal pose, the dynamic surface, which includes clothing, hand gestures as
well as facial expressions. No existing monocular method allows joint tracking
of all these components. To this end, we propose HiFECap, a new neural human
performance capture approach, which simultaneously captures human pose,
clothing, facial expression, and hands just from a single RGB video. We
demonstrate that our proposed network architecture, the carefully designed
training strategy, and the tight integration of parametric face and hand models
to a template mesh enable the capture of all these individual aspects.
Importantly, our method also captures high-frequency details, such as deforming
wrinkles on the clothes, better than the previous works. Furthermore, we show
that HiFECap outperforms the state-of-the-art human performance capture
approaches qualitatively and quantitatively while for the first time capturing
all aspects of the human.

HiFECap 是新的神经人员表现捕捉方法，从单个 RGB 视频中同时捕捉人体姿态、服装、面部表情和手部姿势。该方法表现优于现有表现捕捉方法，首次捕捉到人体的全部细节。