Recent large-scale text-guided diffusion models provide powerful
image-generation capabilities. Currently, a significant effort is given to
enable the modification of these images using text only as means to offer
intuitive and versatile editing. However, editing proves to be difficult for
these generative models due to the inherent nature of editing techniques, which
involves preserving certain content from the original image. Conversely, in
text-based models, even minor modifications to the text prompt frequently
result in an entirely distinct result, making attaining one-shot generation
that accurately corresponds to the users intent exceedingly challenging. In
addition, to edit a real image using these state-of-the-art tools, one must
first invert the image into the pre-trained models domain - adding another
factor affecting the edit quality, as well as latency. In this exploratory
report, we propose LEDITS - a combined lightweight approach for real-image
editing, incorporating the Edit Friendly DDPM inversion technique with Semantic
Guidance, thus extending Semantic Guidance to real image editing, while
harnessing the editing capabilities of DDPM inversion as well. This approach
achieves versatile edits, both subtle and extensive as well as alterations in
composition and style, while requiring no optimization nor extensions to the
architecture.

LEDITS 是一种轻量级的图像编辑方法，结合了 Edit Friendly DDPM 反演技术和语义引导，能够实现对真实图像的细微和广泛编辑，而无需优化和扩展架构。

使用 DDPM 反演和语义指导的实际图像编辑 LEDITS

LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidance

Learning from a large corpus of data, pre-trained models have achieved
impressive progress nowadays. As popular generative pre-training, diffusion
models capture both low-level visual knowledge and high-level semantic
relations. In this paper, we propose to exploit such knowledgeable diffusion
models for mainstream discriminative tasks, i.e., unsupervised object
discovery: saliency segmentation and object localization. However, the
challenges exist as there is one structural difference between generative and
discriminative models, which limits the direct use. Besides, the lack of
explicitly labeled data significantly limits performance in unsupervised
settings. To tackle these issues, we introduce DiffusionSeg, one novel
synthesis-exploitation framework containing two-stage strategies. To alleviate
data insufficiency, we synthesize abundant images, and propose a novel
training-free AttentionCut to obtain masks in the first synthesis stage. In the
second exploitation stage, to bridge the structural gap, we use the inversion
technique, to map the given image back to diffusion features. These features
can be directly used by downstream architectures. Extensive experiments and
ablation studies demonstrate the superiority of adapting diffusion for
unsupervised object discovery.

本文提出了 DiffusionSeg 框架，利用预训练和扩散模型实现无监督物体发现，并采用合成 - 利用两阶段策略来缓解数据不足和结构差异问题，采用反演技术将图像映射回扩散特征，通过大量实验验证了该方法的优越性。

DiffusionSeg: 适应扩散的非监督物体发现

DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discovery

Recent text-guided diffusion models provide powerful image generation
capabilities. Currently, a massive effort is given to enable the modification
of these images using text only as means to offer intuitive and versatile
editing. To edit a real image using these state-of-the-art tools, one must
first invert the image with a meaningful text prompt into the pretrained
model's domain. In this paper, we introduce an accurate inversion technique and
thus facilitate an intuitive text-based modification of the image. Our proposed
inversion consists of two novel key components: (i) Pivotal inversion for
diffusion models. While current methods aim at mapping random noise samples to
a single input image, we use a single pivotal noise vector for each timestamp
and optimize around it. We demonstrate that a direct inversion is inadequate on
its own, but does provide a good anchor for our optimization. (ii) NULL-text
optimization, where we only modify the unconditional textual embedding that is
used for classifier-free guidance, rather than the input text embedding. This
allows for keeping both the model weights and the conditional embedding intact
and hence enables applying prompt-based editing while avoiding the cumbersome
tuning of the model's weights. Our Null-text inversion, based on the publicly
available Stable Diffusion model, is extensively evaluated on a variety of
images and prompt editing, showing high-fidelity editing of real images.

本文介绍了一种精确的图片逆向生成技术，实现了基于文本的图片编辑，通过引入 Pivotal inversion 和 NULL-text optimization 技术，以条件嵌入为导向，避免了模型权重的繁琐调整，并在真实照片上进行了高保真度编辑。