In recent years, the role of image generative models in facial reenactment
has been steadily increasing. Such models are usually subject-agnostic and
trained on domain-wide datasets. The appearance of the reenacted individual is
learned from a single image, and hence, the entire breadth of the individual's
appearance is not entirely captured, leading these methods to resort to
unfaithful hallucination. Thanks to recent advancements, it is now possible to
train a personalized generative model tailored specifically to a given
individual. In this paper, we propose a novel method for facial reenactment
using a personalized generator. We train the generator using frames from a
short, yet varied, self-scan video captured using a simple commodity camera.
Images synthesized by the personalized generator are guaranteed to preserve
identity. The premise of our work is that the task of reenactment is thus
reduced to accurately mimicking head poses and expressions. To this end, we
locate the desired frames in the latent space of the personalized generator
using carefully designed latent optimization. Through extensive evaluation, we
demonstrate state-of-the-art performance for facial reenactment. Furthermore,
we show that since our reenactment takes place in a semantic latent space, it
can be semantically edited and stylized in post-processing.

本文提出利用个性化生成器进行面部复原的新方法，通过采用个性化的生成器，我们可以拍摄一个短且多样化的自我扫描视频以训练我们的生成器，并结合精心设计的潜在优化来确保图像保持身份，并展示我们的方法在面部复原方面具有最先进的性能，同时可以进行语义编辑和风格化处理。

通过个性化生成器进行面部再现

Facial Reenactment Through a Personalized Generator

Discrete variational auto-encoders (VAEs) are able to represent semantic
latent spaces in generative learning. In many real-life settings, the discrete
latent space consists of high-dimensional structures, and propagating gradients
through the relevant structures often requires enumerating over an
exponentially large latent space. Recently, various approaches were devised to
propagate approximated gradients without enumerating over the space of possible
structures. In this work, we use Natural Evolution Strategies (NES), a class of
gradient-free black-box optimization algorithms, to learn discrete structured
VAEs. The NES algorithms are computationally appealing as they estimate
gradients with forward pass evaluations only, thus they do not require to
propagate gradients through their discrete structures. We demonstrate
empirically that optimizing discrete structured VAEs using NES is as effective
as gradient-based approximations. Lastly, we prove NES converges for
non-Lipschitz functions as appear in discrete structured VAEs.

本研究利用自然演化策略算法学习离散结构化变分自编码器（VAE），它们是一种能够在生成学习中表示语义潜在空间的方法，此算法不需要梯度传播，通过前向传递估计梯度；经实验证明，它优化离散结构化 VAE 的效果与基于梯度的算法相当，并证明了在很多情况下 NES 能够收敛于非 Lipschitz 函数。