Despite significant advancements in image customization with diffusion
models, current methods still have several limitations: 1) unintended changes
in non-target areas when regenerating the entire image; 2) guidance solely by a
reference image or text descriptions; and 3) time-consuming fine-tuning, which
limits their practical application. In response, we introduce a tuning-free
framework for simultaneous text-image-guided image customization, enabling
precise editing of specific image regions within seconds. Our approach
preserves the semantic features of the reference image subject while allowing
modification of detailed attributes based on text descriptions. To achieve
this, we propose an innovative attention blending strategy that blends
self-attention features in the UNet decoder during the denoising process. To
our knowledge, this is the first tuning-free method that concurrently utilizes
text and image guidance for image customization in specific regions. Our
approach outperforms previous methods in both human and quantitative
evaluations, providing an efficient solution for various practical
applications, such as image synthesis, design, and creative photography.

通过无需调整的方法，本文提出了一种同时利用文本和图像指导的图像定制方法，允许在几秒钟内精确编辑特定图像区域，通过创新的注意力混合策略，本方法在图像定制中表现优异，是一种高效的解决方案。

基于图像和文本引导的无需调参图像定制

Tuning-Free Image Customization with Image and Text Guidance

Image customization has been extensively studied in text-to-image (T2I)
diffusion models, leading to impressive outcomes and applications. With the
emergence of text-to-video (T2V) diffusion models, its temporal counterpart,
motion customization, has not yet been well investigated. To address the
challenge of one-shot motion customization, we propose Customize-A-Video that
models the motion from a single reference video and adapting it to new subjects
and scenes with both spatial and temporal varieties. It leverages low-rank
adaptation (LoRA) on temporal attention layers to tailor the pre-trained T2V
diffusion model for specific motion modeling from the reference videos. To
disentangle the spatial and temporal information during the training pipeline,
we introduce a novel concept of appearance absorbers that detach the original
appearance from the single reference video prior to motion learning. Our
proposed method can be easily extended to various downstream tasks, including
custom video generation and editing, video appearance customization, and
multiple motion combination, in a plug-and-play fashion. Our project page can
be found at this https URL

通过低秩适应的时空注意力层，该文献介绍了一种用于一次性运动定制的自定义视频模型，可以从单个参考视频中学习运动信息，以适应新的主题和场景，并可轻松扩展到多个下游任务。