Face reenactment methods attempt to restore and re-animate portrait videos as
realistically as possible. Existing methods face a dilemma in quality versus
controllability: 2D GAN-based methods achieve higher image quality but suffer
in fine-grained control of facial attributes compared with 3D counterparts. In
this work, we propose StyleAvatar, a real-time photo-realistic portrait avatar
reconstruction method using StyleGAN-based networks, which can generate
high-fidelity portrait avatars with faithful expression control. We expand the
capabilities of StyleGAN by introducing a compositional representation and a
sliding window augmentation method, which enable faster convergence and improve
translation generalization. Specifically, we divide the portrait scenes into
three parts for adaptive adjustments: facial region, non-facial foreground
region, and the background. Besides, our network leverages the best of UNet,
StyleGAN and time coding for video learning, which enables high-quality video
generation. Furthermore, a sliding window augmentation method together with a
pre-training strategy are proposed to improve translation generalization and
training performance, respectively. The proposed network can converge within
two hours while ensuring high image quality and a forward rendering time of
only 20 milliseconds. Furthermore, we propose a real-time live system, which
further pushes research into applications. Results and experiments demonstrate
the superiority of our method in terms of image quality, full portrait video
generation, and real-time re-animation compared to existing facial reenactment
methods. Training and inference code for this paper are at
this https URL

该论文提出了一种使用 StyleGAN 网络的实时照片级写实肖像化重建方法，可在忠实表情控制下生成高保真肖像化头像，并通过引入组合表示和滑动窗口增强方法将其能力扩展到全身视频生成，使训练和应用的表现和效率得到了提高。