3D-aware image synthesis aims to generate images of objects from multiple
views by learning a 3D representation. However, one key challenge remains:
existing approaches lack geometry constraints, hence usually fail to generate
multi-view consistent images. To address this challenge, we propose Multi-View
Consistent Generative Adversarial Networks (MVCGAN) for high-quality 3D-aware
image synthesis with geometry constraints. By leveraging the underlying 3D
geometry information of generated images, i.e., depth and camera transformation
matrix, we explicitly establish stereo correspondence between views to perform
multi-view joint optimization. In particular, we enforce the photometric
consistency between pairs of views and integrate a stereo mixup mechanism into
the training process, encouraging the model to reason about the correct 3D
shape. Besides, we design a two-stage training strategy with feature-level
multi-view joint optimization to improve the image quality. Extensive
experiments on three datasets demonstrate that MVCGAN achieves the
state-of-the-art performance for 3D-aware image synthesis.

提出了一种基于多视角一致性生成对抗网络（MVCGAN）的高质量 3D 感知图像综合方法，采用几何约束技术，利用深度和相机变换矩阵等信息实现多视角联合优化，同时采用两阶段训练策略和混合训练技术提高模型的生成图像质量，实验结果表明，本方法达到了最先进的性能水平。