We introduce a novel diffusion transformer, LazyDiffusion, that generates
partial image updates efficiently. Our approach targets interactive image
editing applications in which, starting from a blank canvas or an image, a user
specifies a sequence of localized image modifications using binary masks and
text prompts. Our generator operates in two phases. First, a context encoder
processes the current canvas and user mask to produce a compact global context
tailored to the region to generate. Second, conditioned on this context, a
diffusion-based transformer decoder synthesizes the masked pixels in a "lazy"
fashion, i.e., it only generates the masked region. This contrasts with
previous works that either regenerate the full canvas, wasting time and
computation, or confine processing to a tight rectangular crop around the mask,
ignoring the global image context altogether. Our decoder's runtime scales with
the mask size, which is typically small, while our encoder introduces
negligible overhead. We demonstrate that our approach is competitive with
state-of-the-art inpainting methods in terms of quality and fidelity while
providing a 10x speedup for typical user interactions, where the editing mask
represents 10% of the image.

我们介绍了一种新颖的扩散变换器 LazyDiffusion，可高效地生成部分图像更新。我们的方法针对交互式图像编辑应用，其中用户使用二进制掩码和文本提示指定一系列局部图像修改。我们的生成器分为两个阶段：首先，上下文编码器对当前画布和用户掩码进行处理，以生成与待生成区域相适应的简洁全局上下文。其次，在此上下文的条件下，基于扩散的变换器解码器以 “懒惰” 的方式合成掩码像素，即仅生成掩码区域。我们的解码器的运行时间与掩码大小成比例，而我们的编码器几乎不会引入额外开销。我们证明，我们的方法在质量和保真度方面与最先进的修复方法相当，同时为典型的用户交互提供 10 倍的加速，其中编辑掩码占图像的 10%。

懒惰扩散转换器用于交互式图像编辑

Lazy Diffusion Transformer for Interactive Image Editing

Modern text-to-image (T2I) diffusion models can generate images with
remarkable realism and creativity. These advancements have sparked research in
fake image detection and attribution, yet prior studies have not fully explored
the practical and scientific dimensions of this task. In addition to
attributing images to 12 state-of-the-art T2I generators, we provide extensive
analyses on what inference stage hyperparameters and image modifications are
discernible. Our experiments reveal that initialization seeds are highly
detectable, along with other subtle variations in the image generation process
to some extent. We further investigate what visual traces are leveraged in
image attribution by perturbing high-frequency details and employing mid-level
representations of image style and structure. Notably, altering high-frequency
information causes only slight reductions in accuracy, and training an
attributor on style representations outperforms training on RGB images. Our
analyses underscore that fake images are detectable and attributable at various
levels of visual granularity than previously explored.

对现代文本到图像 (T2I) 扩散模型进行了研究，这些模型可以生成非常逼真和具有创造力的图像。我们提供了广泛的分析，关于哪些推理阶段的超参数和图像修改是可辨认的。我们进一步调查了图像归属所依赖的视觉痕迹，通过扰乱高频细节和使用图像风格和结构的中级表示。我们的分析强调，与以前探索的视觉粒度相比，伪造图像在各个层面上都是可检测且可归属的。

在 RGB 及更多领域的文本到图像扩散模型中检测图像归属

Detecting Image Attribution for Text-to-Image Diffusion Models in RGB  and Beyond

As deep neural networks (DNNs) have been integrated into critical systems,
several methods to attack these systems have been developed. These adversarial
attacks make imperceptible modifications to an image that fool DNN classifiers.
We present an adaptive JPEG encoder which defends against many of these
attacks. Experimentally, we show that our method produces images with high
visual quality while greatly reducing the potency of state-of-the-art attacks.
Our algorithm requires only a modest increase in encoding time, produces a
compressed image which can be decompressed by an off-the-shelf JPEG decoder,
and classified by an unmodified classifier

利用自适应 JPEG 编码器来增强深度神经网络对抗性攻击的鲁棒性，不仅可以保持图像的视觉质量，而且无需更改分类器并且压缩图像可以被非定制的 JPEG 解码器解压。