Recently, how to achieve precise image editing has attracted increasing
attention, especially given the remarkable success of text-to-image generation
models. To unify various spatial-aware image editing abilities into one
framework, we adopt the concept of layers from the design domain to manipulate
objects flexibly with various operations. The key insight is to transform the
spatial-aware image editing task into a combination of two sub-tasks:
multi-layered latent decomposition and multi-layered latent fusion. First, we
segment the latent representations of the source images into multiple layers,
which include several object layers and one incomplete background layer that
necessitates reliable inpainting. To avoid extra tuning, we further explore the
inner inpainting ability within the self-attention mechanism. We introduce a
key-masking self-attention scheme that can propagate the surrounding context
information into the masked region while mitigating its impact on the regions
outside the mask. Second, we propose an instruction-guided latent fusion that
pastes the multi-layered latent representations onto a canvas latent. We also
introduce an artifact suppression scheme in the latent space to enhance the
inpainting quality. Due to the inherent modular advantages of such
multi-layered representations, we can achieve accurate image editing, and we
demonstrate that our approach consistently surpasses the latest spatial editing
methods, including Self-Guidance and DiffEditor. Last, we show that our
approach is a unified framework that supports various accurate image editing
tasks on more than six different editing tasks.

通过将空间感知图像编辑任务转化为多层隐变量分解和多层隐变量融合两个子任务，结合遮罩自注意力和抑制伪影技术，我们提出了一种统一的框架，实现精确图像编辑，并证明该方法在各种准确图像编辑任务中的性能超过了最新的空间编辑方法。

DesignEdit: 多层次潜空间分解与融合的统一精准图像编辑

DesignEdit: Multi-Layered Latent Decomposition and Fusion for Unified &  Accurate Image Editing

Text style transfer refers to the task of rephrasing a given text in a
different style. While various methods have been proposed to advance the state
of the art, they often assume the transfer output follows a delta distribution,
and thus their models cannot generate different style transfer results for a
given input text. To address the limitation, we propose a one-to-many text
style transfer framework. In contrast to prior works that learn a one-to-one
mapping that converts an input sentence to one output sentence, our approach
learns a one-to-many mapping that can convert an input sentence to multiple
different output sentences, while preserving the input content. This is
achieved by applying adversarial training with a latent decomposition scheme.
Specifically, we decompose the latent representation of the input sentence to a
style code that captures the language style variation and a content code that
encodes the language style-independent content. We then combine the content
code with the style code for generating a style transfer output. By combining
the same content code with a different style code, we generate a different
style transfer output. Extensive experimental results with comparisons to
several text style transfer approaches on multiple public datasets using a
diverse set of performance metrics validate effectiveness of the proposed
approach.

该论文提出了一种基于一对多映射和潜变量分解的对文本风格转换任务的方法。实验结果表明，该方法在多个公共数据集上的效果明显优于其他文本风格转换方法。

针对输入语句学习生成多个风格转移输出

Learning to Generate Multiple Style Transfer Outputs for an Input  Sentence

Learning generative models that span multiple data modalities, such as vision
and language, is often motivated by the desire to learn more useful,
generalisable representations that faithfully capture common underlying factors
between the modalities. In this work, we characterise successful learning of
such models as the fulfillment of four criteria: i) implicit latent
decomposition into shared and private subspaces, ii) coherent joint generation
over all modalities, iii) coherent cross-generation across individual
modalities, and iv) improved model learning for individual modalities through
multi-modal integration. Here, we propose a mixture-of-experts multimodal
variational autoencoder (MMVAE) to learn generative models on different sets of
modalities, including a challenging image-language dataset, and demonstrate its
ability to satisfy all four criteria, both qualitatively and quantitatively.

本文阐述了成功学习多模态生成模型的四个判定标准，提出了一种混合专家多模态变分自编码器（MMVAE）来学习不同模态的生成模型，并展示了其在具有挑战性的图像 - 语言数据集上实现四个标准的能力，从质量和数量两方面进行了定性和定量。