Image generation has recently seen tremendous advances, with diffusion models
allowing to synthesize convincing images for a large variety of text prompts.
In this article, we propose DiffEdit, a method to take advantage of
text-conditioned diffusion models for the task of semantic image editing, where
the goal is to edit an image based on a text query. Semantic image editing is
an extension of image generation, with the additional constraint that the
generated image should be as similar as possible to a given input image.
Current editing methods based on diffusion models usually require to provide a
mask, making the task much easier by treating it as a conditional inpainting
task. In contrast, our main contribution is able to automatically generate a
mask highlighting regions of the input image that need to be edited, by
contrasting predictions of a diffusion model conditioned on different text
prompts. Moreover, we rely on latent inference to preserve content in those
regions of interest and show excellent synergies with mask-based diffusion.
DiffEdit achieves state-of-the-art editing performance on ImageNet. In
addition, we evaluate semantic image editing in more challenging settings,
using images from the COCO dataset as well as text-based generated images.

本文提出了一种基于条件扩散模型的语义图像编辑方法 DiffEdit，能够自动生成需要编辑的图像区域的遮罩，并利用潜在推理保留感兴趣区域的内容，该方法在 ImageNet 数据集上实现了最先进的编辑表现。