Despite the ability of existing large-scale text-to-image (T2I) models to
generate high-quality images from detailed textual descriptions, they often
lack the ability to precisely edit the generated or real images. In this paper,
we propose a novel image editing method, DragonDiffusion, enabling Drag-style
manipulation on Diffusion models. Specifically, we construct classifier
guidance based on the strong correspondence of intermediate features in the
diffusion model. It can transform the editing signals into gradients via
feature correspondence loss to modify the intermediate representation of the
diffusion model. Based on this guidance strategy, we also build a multi-scale
guidance to consider both semantic and geometric alignment. Moreover, a
cross-branch self-attention is added to maintain the consistency between the
original image and the editing result. Our method, through an efficient design,
achieves various editing modes for the generated or real images, such as object
moving, object resizing, object appearance replacement, and content dragging.
It is worth noting that all editing and content preservation signals come from
the image itself, and the model does not require fine-tuning or additional
modules. Our source code will be available at
this https URL

本文介绍了一种基于 DragonDiffusion 的图像编辑方法，通过构建一个分类器指导策略，将编辑信号转化为梯度并利用特征对应损失修改分布式模型的中间表示，在保持原图像和编辑结果的一致性的同时，实现各种编辑模式，诸如物体移动、物体缩放、物体外观替换和内容拖动。

DragonDiffusion：在扩散模型上实现拖拽式操作

DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models

Intermediate features of a pre-trained model have been shown informative for
making accurate predictions on downstream tasks, even if the model backbone is
kept frozen. The key challenge is how to utilize these intermediate features
given their gigantic amount. We propose visual query tuning (VQT), a simple yet
effective approach to aggregate intermediate features of Vision Transformers.
Through introducing a handful of learnable ``query'' tokens to each layer, VQT
leverages the inner workings of Transformers to ``summarize'' rich intermediate
features of each layer, which can then be used to train the prediction heads of
downstream tasks. As VQT keeps the intermediate features intact and only learns
to combine them, it enjoys memory efficiency in training, compared to many
other parameter-efficient fine-tuning approaches that learn to adapt features
and need back-propagation through the entire backbone. This also suggests the
complementary role between VQT and those approaches in transfer learning.
Empirically, VQT consistently surpasses the state-of-the-art approach that
utilizes intermediate features for transfer learning and outperforms full
fine-tuning in many cases. Compared to parameter-efficient approaches that
adapt features, VQT achieves much higher accuracy under memory constraints.
Most importantly, VQT is compatible with these approaches to attain even higher
accuracy, making it a simple add-on to further boost transfer learning.

介绍了一种简单而有效的方法，“视觉查询调整 (VQT)”，可以聚合视觉变换器的中间特征以用于传输学习，同时保持了中间特征的完整性和内存效率。与其他参数有效的微调方法相比，VQT 在内存限制下获得更高的准确度，并且可以与这些方法兼容以实现更高的准确度。

视觉查询调整：实现中间表示的有效使用，以实现参数和内存高效的迁移学习

Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning

Recent developments in deep domain adaptation have allowed knowledge transfer
from a labeled source domain to an unlabeled target domain at the level of
intermediate features or input pixels. We propose that advantages may be
derived by combining them, in the form of different insights that lead to a
novel design and complementary properties that result in better performance. At
the feature level, inspired by insights from semi-supervised learning, we
propose a classification-aware domain adversarial neural network that brings
target examples into more classifiable regions of source domain. Next, we posit
that computer vision insights are more amenable to injection at the pixel
level. In particular, we use 3D geometry and image synthesis based on a
generalized appearance flow to preserve identity across pose transformations,
while using an attribute-conditioned CycleGAN to translate a single source into
multiple target images that differ in lower-level properties such as lighting.
Besides standard UDA benchmark, we validate on a novel and apt problem of car
recognition in unlabeled surveillance images using labeled images from the web,
handling explicitly specified, nameable factors of variation through
pixel-level and implicit, unspecified factors through feature-level adaptation.

通过深度域自适应进行知识迁移来提高性能，本研究提出了利用半监督学习洞察力设计的分类感知域对抗神经网络以在源域中将目标例子转化为更可分类的区域，并利用建立在通用外观流的三维几何与图像合成的基础上的属性条件换档深度生成对抗网络将源图像转化为多目标图像的意见。