Text guided image editing on real images given only the image and the target
text prompt as inputs, is a very general and challenging problem, which
requires the editing model to reason by itself which part of the image should
be edited, to preserve the characteristics of original image, and also to
perform complicated non-rigid editing. Previous fine-tuning based solutions are
time-consuming and vulnerable to overfitting, limiting their editing
capabilities. To tackle these issues, we design a novel text guided image
editing method, Forgedit. First, we propose a novel fine-tuning framework which
learns to reconstruct the given image in less than one minute by vision
language joint learning. Then we introduce vector subtraction and vector
projection to explore the proper text embedding for editing. We also find a
general property of UNet structures in Diffusion Models and inspired by such a
finding, we design forgetting strategies to diminish the fatal overfitting
issues and significantly boost the editing abilities of Diffusion Models. Our
method, Forgedit, implemented with Stable Diffusion, achieves new
state-of-the-art results on the challenging text guided image editing benchmark
TEdBench, surpassing the previous SOTA method Imagic with Imagen, in terms of
both CLIP score and LPIPS score. Codes are available at
this https URL

通过图像重建、文本嵌入以及使用 UNet 结构和扩散模型，我们设计了一种新的文本引导图像编辑方法 Forgedit，具有强大的编辑能力，并在具有挑战性的文本引导图像编辑基准测试 TEdBench 中超越了以往的方法 Imagic，实现了最新的 state-of-the-art 结果。