Text-conditioned diffusion models can generate impressive images, but fall
short when it comes to fine-grained control. Unlike direct-editing tools like
Photoshop, text conditioned models require the artist to perform "prompt
engineering," constructing special text sentences to control the style or
amount of a particular subject present in the output image. Our goal is to
provide fine-grained control over the style and substance specified by the
prompt, for example to adjust the intensity of styles in different regions of
the image (Figure 1). Our approach is to decompose the text prompt into
conceptual elements, and apply a separate guidance term for each element in a
single diffusion process. We introduce guidance scale functions to control when
in the diffusion process and \emph{where} in the image to intervene. Since the
method is based solely on adjusting diffusion guidance, it does not require
fine-tuning or manipulating the internal layers of the diffusion model's neural
network, and can be used in conjunction with LoRA- or DreamBooth-trained models
(Figure2). Project page: this https URL

通过分解文本提示为概念元素，并在单个扩散过程中应用单独的指导项，我们可以提供对风格和内容的细粒度控制，同时不需要微调扩散模型的内部层或操纵其神经网络。

DreamWalk: 使用扩散引导的风格空间探索

DreamWalk: Style Space Exploration using Diffusion Guidance

With the great success of text-conditioned diffusion models in creative
text-to-image generation, various text-driven image editing approaches have
attracted the attentions of many researchers. However, previous works mainly
focus on discreteness-sensitive instructions such as adding, removing or
replacing specific objects, background elements or global styles (i.e., hard
editing), while generally ignoring subject-binding but semantically
fine-changing continuity-sensitive instructions such as actions, poses or
adjectives, and so on (i.e., soft editing), which hampers generative AI from
generating user-customized visual contents. To mitigate this predicament, we
propose a spatio-temporal guided adaptive editing algorithm AdapEdit, which
realizes adaptive image editing by introducing a soft-attention strategy to
dynamically vary the guiding degree from the editing conditions to visual
pixels from both temporal and spatial perspectives. Note our approach has a
significant advantage in preserving model priors and does not require model
training, fine-tuning, extra data, or optimization. We present our results over
a wide variety of raw images and editing instructions, demonstrating
competitive performance and showing it significantly outperforms the previous
approaches.

为了生成用户定制的视觉内容，本文提出了一种时空引导自适应编辑算法 AdapEdit，通过引入软注意力策略从时空角度动态变化编辑条件对视觉像素的引导程度，成功实现了自适应图像编辑，并在各种原始图像和编辑指令上展示了竞争性能并显著超越了以前的方法。

AdapEdit: 基于时空引导的自适应编辑算法，用于基于文本连续感知的图像编辑

AdapEdit: Spatio-Temporal Guided Adaptive Editing Algorithm for  Text-Based Continuity-Sensitive Image Editing

Text-conditioned diffusion models have emerged as a promising tool for neural
video generation. However, current models still struggle with intricate
spatiotemporal prompts and often generate restricted or incorrect motion (e.g.,
even lacking the ability to be prompted for objects moving from left to right).
To address these limitations, we introduce LLM-grounded Video Diffusion (LVD).
Instead of directly generating videos from the text inputs, LVD first leverages
a large language model (LLM) to generate dynamic scene layouts based on the
text inputs and subsequently uses the generated layouts to guide a diffusion
model for video generation. We show that LLMs are able to understand complex
spatiotemporal dynamics from text alone and generate layouts that align closely
with both the prompts and the object motion patterns typically observed in the
real world. We then propose to guide video diffusion models with these layouts
by adjusting the attention maps. Our approach is training-free and can be
integrated into any video diffusion model that admits classifier guidance. Our
results demonstrate that LVD significantly outperforms its base video diffusion
model and several strong baseline methods in faithfully generating videos with
the desired attributes and motion patterns.

在视频生成方面，本研究提出了一种基于文本的扩散模型，通过使用大型语言模型（LLM）生成的动态场景布局来引导视频生成过程，旨在解决现有模型在处理复杂时空提示时遇到的限制和错误运动的问题。实验证明该方法在生成具有所需特征和运动模式的视频方面显著优于基准模型和一些强基线方法。