Large-scale text-to-image models have demonstrated amazing ability to
synthesize diverse and high-fidelity images. However, these models are often
violated by several limitations. Firstly, they require the user to provide
precise and contextually relevant descriptions for the desired image
modifications. Secondly, current models can impose significant changes to the
original image content during the editing process. In this paper, we explore
ReGeneration learning in an image-to-image Diffusion model (ReDiffuser), that
preserves the content of the original image without human prompting and the
requisite editing direction is automatically discovered within the text
embedding space. To ensure consistent preservation of the shape during image
editing, we propose cross-attention guidance based on regeneration learning.
This novel approach allows for enhanced expression of the target domain
features while preserving the original shape of the image. In addition, we
introduce a cooperative update strategy, which allows for efficient
preservation of the original shape of an image, thereby improving the quality
and consistency of shape preservation throughout the editing process. Our
proposed method leverages an existing pre-trained text-image diffusion model
without any additional training. Extensive experiments show that the proposed
method outperforms existing work in both real and synthetic image editing.

本文提出了一种 ReDiffuser 图片修复模型，该模型使用自动生成的描述来完成图片编辑，利用再生学习和交叉注意力向导实现图片对形状的一致性保留，并引入一种协作更新策略，提高图片修复的质量和一致性。实验结果表明，该方法在真实和合成图片编辑方面优于现有方法。

使用丰富提示进行零样本图像转换的扩散模型再生学习

ReGeneration Learning of Diffusion Models with Rich Prompts for  Zero-Shot Image Translation

Machine learning methods for conditional data generation usually build a
mapping from source conditional data X to target data Y. The target Y (e.g.,
text, speech, music, image, video) is usually high-dimensional and complex, and
contains information that does not exist in source data, which hinders
effective and efficient learning on the source-target mapping. In this paper,
we present a learning paradigm called regeneration learning for data
generation, which first generates Y' (an abstraction/representation of Y) from
X and then generates Y from Y'. During training, Y' is obtained from Y through
either handcrafted rules or self-supervised learning and is used to learn
X-->Y' and Y'-->Y. Regeneration learning extends the concept of representation
learning to data generation tasks, and can be regarded as a counterpart of
traditional representation learning, since 1) regeneration learning handles the
abstraction (Y') of the target data Y for data generation while traditional
representation learning handles the abstraction (X') of source data X for data
understanding; 2) both the processes of Y'-->Y in regeneration learning and
X-->X' in representation learning can be learned in a self-supervised way
(e.g., pre-training); 3) both the mappings from X to Y' in regeneration
learning and from X' to Y in representation learning are simpler than the
direct mapping from X to Y. We show that regeneration learning can be a
widely-used paradigm for data generation (e.g., text generation, speech
recognition, speech synthesis, music composition, image generation, and video
generation) and can provide valuable insights into developing data generation
methods.

本文介绍了一种称为再生学习的学习范式，将表示学习的概念扩展到数据生成任务，使用 Y' 进行数据生成。再生学习作为传统表示学习的对应物，可以广泛应用于数据生成，并且可以为开发数据生成方法提供有价值的见解。