3D-consistent image generation from a single 2D semantic label is an
important and challenging research topic in computer graphics and computer
vision. Although some related works have made great progress in this field,
most of the existing methods suffer from poor disentanglement performance of
shape and appearance, and lack multi-modal control. In this paper, we propose a
novel end-to-end 3D-aware image generation and editing model incorporating
multiple types of conditional inputs, including pure noise, text and reference
image. On the one hand, we dive into the latent space of 3D Generative
Adversarial Networks (GANs) and propose a novel disentanglement strategy to
separate appearance features from shape features during the generation process.
On the other hand, we propose a unified framework for flexible image generation
and editing tasks with multi-modal conditions. Our method can generate diverse
images with distinct noises, edit the attribute through a text description and
conduct style transfer by giving a reference RGB image. Extensive experiments
demonstrate that the proposed method outperforms alternative approaches both
qualitatively and quantitatively on image generation and editing.

本文提出了一种新颖的端到端 3D 感知图像生成和编辑模型，通过纯噪声、文本和参考图像等多种条件输入，在 3D 生成对抗网络（GANs）的潜在空间中深入研究并提出解缠特性较好的生成策略，同时采用统一框架进行灵活的图像生成和编辑任务，实现多模态条件下的多样图像生成、属性编辑和风格迁移。广泛实验证明，该方法在图像生成和编辑方面在质量和数量上均优于替代方法。