Precise image editing with text-to-image models has attracted increasing
interest due to their remarkable generative capabilities and user-friendly
nature. However, such attempts face the pivotal challenge of misalignment
between the intended precise editing target regions and the broader area
impacted by the guidance in practice. Despite excellent methods leveraging
attention mechanisms that have been developed to refine the editing guidance,
these approaches necessitate modifications through complex network architecture
and are limited to specific editing tasks. In this work, we re-examine the
diffusion process and misalignment problem from a frequency perspective,
revealing that, due to the power law of natural images and the decaying noise
schedule, the denoising network primarily recovers low-frequency image
components during the earlier timesteps and thus brings excessive low-frequency
signals for editing. Leveraging this insight, we introduce a novel fine-tuning
free approach that employs progressive $\textbf{Fre}$qu$\textbf{e}$ncy
truncation to refine the guidance of $\textbf{Diff}$usion models for universal
editing tasks ($\textbf{FreeDiff}$). Our method achieves comparable results
with state-of-the-art methods across a variety of editing tasks and on a
diverse set of images, highlighting its potential as a versatile tool in image
editing applications.

通过频率截断的方法，我们提出了一种新颖的 fine-tuning 自由方法用于改善扩散模型的引导，从而实现了在各种编辑任务和不同图像集上与最先进方法相媲美的结果，突显了它在图像编辑应用中的多功能潜力。

FreeDiff: 基于扩散模型的图像编辑的递进频率截断

FreeDiff: Progressive Frequency Truncation for Image Editing with  Diffusion Models

Recently, how to achieve precise image editing has attracted increasing
attention, especially given the remarkable success of text-to-image generation
models. To unify various spatial-aware image editing abilities into one
framework, we adopt the concept of layers from the design domain to manipulate
objects flexibly with various operations. The key insight is to transform the
spatial-aware image editing task into a combination of two sub-tasks:
multi-layered latent decomposition and multi-layered latent fusion. First, we
segment the latent representations of the source images into multiple layers,
which include several object layers and one incomplete background layer that
necessitates reliable inpainting. To avoid extra tuning, we further explore the
inner inpainting ability within the self-attention mechanism. We introduce a
key-masking self-attention scheme that can propagate the surrounding context
information into the masked region while mitigating its impact on the regions
outside the mask. Second, we propose an instruction-guided latent fusion that
pastes the multi-layered latent representations onto a canvas latent. We also
introduce an artifact suppression scheme in the latent space to enhance the
inpainting quality. Due to the inherent modular advantages of such
multi-layered representations, we can achieve accurate image editing, and we
demonstrate that our approach consistently surpasses the latest spatial editing
methods, including Self-Guidance and DiffEditor. Last, we show that our
approach is a unified framework that supports various accurate image editing
tasks on more than six different editing tasks.

通过将空间感知图像编辑任务转化为多层隐变量分解和多层隐变量融合两个子任务，结合遮罩自注意力和抑制伪影技术，我们提出了一种统一的框架，实现精确图像编辑，并证明该方法在各种准确图像编辑任务中的性能超过了最新的空间编辑方法。