Researchers have recently begun exploring the use of StyleGAN-based models
for real image editing. One particularly interesting application is using
natural language descriptions to guide the editing process. Existing approaches
for editing images using language either resort to instance-level latent code
optimization or map predefined text prompts to some editing directions in the
latent space. However, these approaches have inherent limitations. The former
is not very efficient, while the latter often struggles to effectively handle
multi-attribute changes. To address these weaknesses, we present CLIPInverter,
a new text-driven image editing approach that is able to efficiently and
reliably perform multi-attribute changes. The core of our method is the use of
novel, lightweight text-conditioned adapter layers integrated into pretrained
GAN-inversion networks. We demonstrate that by conditioning the initial
inversion step on the CLIP embedding of the target description, we are able to
obtain more successful edit directions. Additionally, we use a CLIP-guided
refinement step to make corrections in the resulting residual latent codes,
which further improves the alignment with the text prompt. Our method
outperforms competing approaches in terms of manipulation accuracy and
photo-realism on various domains including human faces, cats, and birds, as
shown by our qualitative and quantitative results.

本文提出了一种新的基于文本的图像编辑方法 CLIPInverter，通过在预训练的 GAN 反演网络中集成轻量级文本适配器层，以目标描述的 CLIP 嵌入为条件进行初始反演步骤的条件化，通过使用 CLIP 引导的细化步骤来对结果残留潜在编码进行更正，从而实现了高效稳定地进行多属性更改，因此在各个领域，包括人类脸部，猫和鸟类方面，我们的方法在操作精度和真实度方面均表现优异。