Although 2D generative models have made great progress in face image
generation and animation, they often suffer from undesirable artifacts such as
3D inconsistency when rendering images from different camera viewpoints. This
prevents them from synthesizing video animations indistinguishable from real
ones. Recently, 3D-aware GANs extend 2D GANs for explicit disentanglement of
camera pose by leveraging 3D scene representations. These methods can well
preserve the 3D consistency of the generated images across different views, yet
they cannot achieve fine-grained control over other attributes, among which
facial expression control is arguably the most useful and desirable for face
animation. In this paper, we propose an animatable 3D-aware GAN for multiview
consistent face animation generation. The key idea is to decompose the 3D
representation of the 3D-aware GAN into a template field and a deformation
field, where the former represents different identities with a canonical
expression, and the latter characterizes expression variations of each
identity. To achieve meaningful control over facial expressions via
deformation, we propose a 3D-level imitative learning scheme between the
generator and a parametric 3D face model during adversarial training of the
3D-aware GAN. This helps our method achieve high-quality animatable face image
generation with strong visual 3D consistency, even though trained with only
unstructured 2D images. Extensive experiments demonstrate our superior
performance over prior works. Project page:
this https URL

本研究提出一种可动画的 3D 感知生成对抗网络用于多视角一致的人脸动画生成，通过将 3D 感知生成对抗网络的 3D 表示分解为模板场和形变场，并提出 3D 级模仿学习方案，帮助我们实现高质量的动画面部图像生成并具有较强的可视化 3D 一致性。