Recent advances in vision-language models like Stable Diffusion have shown
remarkable power in creative image synthesis and editing.However, most existing
text-to-image editing methods encounter two obstacles: First, the text prompt
needs to be carefully crafted to achieve good results, which is not intuitive
or user-friendly. Second, they are insensitive to local edits and can
irreversibly affect non-edited regions, leaving obvious editing traces. To
tackle these problems, we propose a Zero-shot instructiON-guided local image
Editing approach, termed ZONE. We first convert the editing intent from the
user-provided instruction (e.g., ``make his tie blue") into specific image
editing regions through InstructPix2Pix. We then propose a Region-IoU scheme
for precise image layer extraction from an off-the-shelf segment model. We
further develop an edge smoother based on FFT for seamless blending between the
layer and the image.Our method allows for arbitrary manipulation of a specific
region with a single instruction while preserving the rest. Extensive
experiments demonstrate that our ZONE achieves remarkable local editing results
and user-friendliness, outperforming state-of-the-art methods.

基于 Zero-shot 指令引导的局部图像编辑方法 ZONE 通过转换用户提供的指令，实现了对特定图像区域的任意操作，同时保留其他区域的编辑友好性，展现了出色的局部编辑效果，优于现有的最先进方法。

区域：零样本指导下的局部编辑

ZONE: Zero-Shot Instruction-Guided Local Editing

The tremendous progress in neural image generation, coupled with the
emergence of seemingly omnipotent vision-language models has finally enabled
text-based interfaces for creating and editing images. Handling generic images
requires a diverse underlying generative model, hence the latest works utilize
diffusion models, which were shown to surpass GANs in terms of diversity. One
major drawback of diffusion models, however, is their relatively slow inference
time. In this paper, we present an accelerated solution to the task of local
text-driven editing of generic images, where the desired edits are confined to
a user-provided mask. Our solution leverages a recent text-to-image Latent
Diffusion Model (LDM), which speeds up diffusion by operating in a
lower-dimensional latent space. We first convert the LDM into a local image
editor by incorporating Blended Diffusion into it. Next we propose an
optimization-based solution for the inherent inability of this LDM to
accurately reconstruct images. Finally, we address the scenario of performing
local edits using thin masks. We evaluate our method against the available
baselines both qualitatively and quantitatively and demonstrate that in
addition to being faster, our method achieves better precision than the
baselines while mitigating some of their artifacts. Project page is available
at this https URL

本研究介绍了一种基于文本的图像编辑方法，利用最新的扩散模型对一些基于文本主题的地方图像进行编辑，通过结合扩散模型的速度和 Blended Diffusion，提高了编辑的效率，并通过优化方法来解决扩散模型无法完美重建图像的问题，实现了比当前方法更高的精度和速度