With the recent advances in NeRF-based 3D aware GANs quality, projecting an
image into the latent space of these 3D-aware GANs has a natural advantage over
2D GAN inversion: not only does it allow multi-view consistent editing of the
projected image, but it also enables 3D reconstruction and novel view synthesis
when given only a single image. However, the explicit viewpoint control acts as
a main hindrance in the 3D GAN inversion process, as both camera pose and
latent code have to be optimized simultaneously to reconstruct the given image.
Most works that explore the latent space of the 3D-aware GANs rely on
ground-truth camera viewpoint or deformable 3D model, thus limiting their
applicability. In this work, we introduce a generalizable 3D GAN inversion
method that infers camera viewpoint and latent code simultaneously to enable
multi-view consistent semantic image editing. The key to our approach is to
leverage pre-trained estimators for better initialization and utilize the
pixel-wise depth calculated from NeRF parameters to better reconstruct the
given image. We conduct extensive experiments on image reconstruction and
editing both quantitatively and qualitatively, and further compare our results
with 2D GAN-based editing to demonstrate the advantages of utilizing the latent
space of 3D GANs. Additional results and visualizations are available at
this https URL .

本研究提出了一种通用的 3D GAN 反演方法，可以同时计算相机视角和潜变量代码，从而实现多视角一致的语义图像编辑。其关键在于利用预训练的估算器用于更好的初始化，并利用从 NeRF 参数计算的像素深度来更好地重构给定的图像。