There are many recent research efforts to fine-tune a pre-trained generator
with a few target images to generate images of a novel domain. Unfortunately,
these methods often suffer from overfitting or under-fitting when fine-tuned
with a single target image. To address this, here we present a novel
single-shot GAN adaptation method through unified CLIP space manipulations.
Specifically, our model employs a two-step training strategy: reference image
search in the source generator using a CLIP-guided latent optimization,
followed by generator fine-tuning with a novel loss function that imposes CLIP
space consistency between the source and adapted generators. To further improve
the adapted model to produce spatially consistent samples with respect to the
source generator, we also propose contrastive regularization for patchwise
relationships in the CLIP space. Experimental results show that our model
generates diverse outputs with the target texture and outperforms the baseline
models both qualitatively and quantitatively. Furthermore, we show that our
CLIP space manipulation strategy allows more effective attribute editing.

本研究提出了一种采用统一 CLIP 空间变换的单次生成器适应方法，该方法采用基于 CLIP 导向的潜变量优化进行参考图像搜索，并使用新的损失功能进行生成器微调以实现 CLIP 空间一致性。此外，研究者还提出了一种面向 CLIP 空间的路径关系对比正则化方法，以进一步提高生成模型的效果。该方法能够有效地进行属性编辑且具有较好的实验结果。