NeRF-based 3D-aware Generative Adversarial Networks (GANs) like EG3D or
GIRAFFE have shown very high rendering quality under large representational
variety. However, rendering with Neural Radiance Fields poses challenges for 3D
applications: First, the significant computational demands of NeRF rendering
preclude its use on low-power devices, such as mobiles and VR/AR headsets.
Second, implicit representations based on neural networks are difficult to
incorporate into explicit 3D scenes, such as VR environments or video games. 3D
Gaussian Splatting (3DGS) overcomes these limitations by providing an explicit
3D representation that can be rendered efficiently at high frame rates. In this
work, we present a novel approach that combines the high rendering quality of
NeRF-based 3D-aware GANs with the flexibility and computational advantages of
3DGS. By training a decoder that maps implicit NeRF representations to explicit
3D Gaussian Splatting attributes, we can integrate the representational
diversity and quality of 3D GANs into the ecosystem of 3D Gaussian Splatting
for the first time. Additionally, our approach allows for a high resolution GAN
inversion and real-time GAN editing with 3D Gaussian Splatting scenes.

本文介绍了一种将 NeRF-based 3D-aware GANs 的高渲染质量与 3D Gaussian Splatting 的灵活性和计算优势相结合的新方法。通过训练一个解码器，将隐式的 NeRF 表示映射到显式的 3D Gaussian Splatting 属性，我们可以首次将 3D GANs 的代表性多样性和质量整合到 3D Gaussian Splatting 的生态系统中。此外，我们的方法还允许高分辨率 GAN 反演和实时 GAN 编辑与 3D Gaussian Splatting 场景的结合。