Despite many attempts to leverage pre-trained text-to-image models (T2I) like
Stable Diffusion (SD) for controllable image editing, producing good
predictable results remains a challenge. Previous approaches have focused on
either fine-tuning pre-trained T2I models on specific datasets to generate
certain kinds of images (e.g., with a specific object or person), or on
optimizing the weights, text prompts, and/or learning features for each input
image in an attempt to coax the image generator to produce the desired result.
However, these approaches all have shortcomings and fail to produce good
results in a predictable and controllable manner. To address this problem, we
present TiNO-Edit, an SD-based method that focuses on optimizing the noise
patterns and diffusion timesteps during editing, something previously
unexplored in the literature. With this simple change, we are able to generate
results that both better align with the original images and reflect the desired
result. Furthermore, we propose a set of new loss functions that operate in the
latent domain of SD, greatly speeding up the optimization when compared to
prior approaches, which operate in the pixel domain. Our method can be easily
applied to variations of SD including Textual Inversion and DreamBooth that
encode new concepts and incorporate them into the edited results. We present a
host of image-editing capabilities enabled by our approach. Our code is
publicly available at this https URL

通过对噪音模式和扩散时间步进行优化，本文提出了基于 SD 的 TiNO-Edit 方法，其能够生成与原始图像更加吻合且符合预期结果的编辑结果，同时在 SD 的潜在域中提出了一组新的损失函数来加速优化过程。