Portrait stylization is a long-standing task enabling extensive applications.
Although 2D-based methods have made great progress in recent years, real-world
applications such as metaverse and games often demand 3D content. On the other
hand, the requirement of 3D data, which is costly to acquire, significantly
impedes the development of 3D portrait stylization methods. In this paper,
inspired by the success of 3D-aware GANs that bridge 2D and 3D domains with 3D
fields as the intermediate representation for rendering 2D images, we propose a
novel method, dubbed HyperStyle3D, based on 3D-aware GANs for 3D portrait
stylization. At the core of our method is a hyper-network learned to manipulate
the parameters of the generator in a single forward pass. It not only offers a
strong capacity to handle multiple styles with a single model, but also enables
flexible fine-grained stylization that affects only texture, shape, or local
part of the portrait. While the use of 3D-aware GANs bypasses the requirement
of 3D data, we further alleviate the necessity of style images with the CLIP
model being the stylization guidance. We conduct an extensive set of
experiments across the style, attribute, and shape, and meanwhile, measure the
3D consistency. These experiments demonstrate the superior capability of our
HyperStyle3D model in rendering 3D-consistent images in diverse styles,
deforming the face shape, and editing various attributes.

本文提出了一种基于 3D-aware GANs 的 HyperStyle3D 方法，利用 CLIP 模型作为样式指导，可以在不要求 3D 数据的情况下进行立体画风化处理，实验结果表明该方法在不同的画风、形状变形、属性修改等方面展现了优异表现，且保持了 3D 一致性。