Existing 3D-aware portrait synthesis methods can generate impressive
high-quality images while preserving strong 3D consistency. However, most of
them cannot support the fine-grained part-level control over synthesized
images. Conversely, some GAN-based 2D portrait synthesis methods can achieve
clear disentanglement of facial regions, but they cannot preserve view
consistency due to a lack of 3D modeling abilities. To address these issues, we
propose 3D-SSGAN, a novel framework for 3D-aware compositional portrait image
synthesis. First, a simple yet effective depth-guided 2D-to-3D lifting module
maps the generated 2D part features and semantics to 3D. Then, a volume
renderer with a novel 3D-aware semantic mask renderer is utilized to produce
the composed face features and corresponding masks. The whole framework is
trained end-to-end by discriminating between real and synthesized 2D images and
their semantic masks. Quantitative and qualitative evaluations demonstrate the
superiority of 3D-SSGAN in controllable part-level synthesis while preserving
3D view consistency.

3D-SSGAN 是一个新颖的 3D 感知合成肖像图像的框架，通过提供细粒度的部件级控制，并保持 3D 视图的一致性，在合成肖像图像的同时实现了清晰的人脸区域解缠，其通过引入一个基于深度引导的 2D 到 3D 映射模块和一个具有新颖的 3D 感知语义掩码渲染器的体积渲染器来生成合成的人脸特征和相应的掩码，并通过区分真实和合成的 2D 图像和语义掩码的方式进行端到端的训练，定量和定性评估证明了 3D-SSGAN 在可控的部件级合成和保持 3D 视图一致性方面的优越性。