Diffusion-based image translation guided by semantic texts or a single target
image has enabled flexible style transfer which is not limited to the specific
domains. Unfortunately, due to the stochastic nature of diffusion models, it is
often difficult to maintain the original content of the image during the
reverse diffusion. To address this, here we present a novel diffusion-based
unsupervised image translation method using disentangled style and content
representation.
Specifically, inspired by the splicing Vision Transformer, we extract
intermediate keys of multihead self attention layer from ViT model and used
them as the content preservation loss. Then, an image guided style transfer is
performed by matching the [CLS] classification token from the denoised samples
and target image, whereas additional CLIP loss is used for the text-driven
style transfer. To further accelerate the semantic change during the reverse
diffusion, we also propose a novel semantic divergence loss and resampling
strategy. Our experimental results show that the proposed method outperforms
state-of-the-art baseline models in both text-guided and image-guided
translation tasks.

本文提出了一种基于扩散的非监督图像转换方法，使用分离的风格和内容表征，并使用 ViT 模型中的中间键提取多头自注意层作为内容保存损失，并匹配文本驱动风格转移的 [CLS] 分类令牌，同时使用额外的 CLIP 损失，实验结果表明，该方法在文本引导和图像引导的转换任务中优于现有的基准模型。

基于扩散的图像翻译：使用解缕分离的风格和内容表征

Diffusion-based Image Translation using Disentangled Style and Content Representation

Image to image translation aims to learn a mapping that transforms an image
from one visual domain to another. Recent works assume that images descriptors
can be disentangled into a domain-invariant content representation and a
domain-specific style representation. Thus, translation models seek to preserve
the content of source images while changing the style to a target visual
domain. However, synthesizing new images is extremely challenging especially in
multi-domain translations, as the network has to compose content and style to
generate reliable and diverse images in multiple domains. In this paper we
propose the use of an image retrieval system to assist the image-to-image
translation task. First, we train an image-to-image translation model to map
images to multiple domains. Then, we train an image retrieval model using real
and generated images to find images similar to a query one in content but in a
different domain. Finally, we exploit the image retrieval system to fine-tune
the image-to-image translation model and generate higher quality images. Our
experiments show the effectiveness of the proposed solution and highlight the
contribution of the retrieval network, which can benefit from additional
unlabeled data and help image-to-image translation models in the presence of
scarce data.

本文提出使用图像检索系统辅助图像翻译，从而生成更高质量的图片，实验结果表明该方案是有效的。

基于检索引导的无监督多领域图像到图像翻译

Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation

We consider the disentanglement of the representations of the relevant
attributes of the data (content) from all other factors of variations (style)
using Variational Autoencoders. Some recent works addressed this problem by
utilizing grouped observations, where the content attributes are assumed to be
common within each group, while there is no any supervised information on the
style factors. In many cases, however, these methods fail to prevent the models
from using the style variables to encode content related features as well. This
work supplements these algorithms with a method that eliminates the content
information in the style representations. For that purpose the training
objective is augmented to minimize an appropriately defined mutual information
term in an adversarial way. Experimental results and comparisons on image
datasets show that the resulting method can efficiently separate the content
and style related attributes and generalizes to unseen data.

使用变分自编码器 (Variational Autoencoders)，通过增加最小化对抗互信息项来消除样式 (Syle) 表示中的内容 (Content) 信息，从而实现有效地分离内容和样式相关属性。

组合观察的对抗性解缠

Adversarial Disentanglement with Grouped Observations

Generating an abstract from a collection of documents is a desirable
capability for many real-world applications. However, abstractive approaches to
multi-document summarization have not been thoroughly investigated. This paper
studies the feasibility of using Abstract Meaning Representation (AMR), a
semantic representation of natural language grounded in linguistic theory, as a
form of content representation. Our approach condenses source documents to a
set of summary graphs following the AMR formalism. The summary graphs are then
transformed to a set of summary sentences in a surface realization step. The
framework is fully data-driven and flexible. Each component can be optimized
independently using small-scale, in-domain training data. We perform
experiments on benchmark summarization datasets and report promising results.
We also describe opportunities and challenges for advancing this line of
research.

本文研究将多篇文档进行内容归纳的可行性，并提出了使用 Abstract Meaning Representation（AMR）作为形式化内容表示的方法，通过将源文档压缩为 AMR 标准的摘要图，再将摘要图转换为一组摘要句子的表层实现步骤。实验结果表明，该方法具有良好的潜力，并描述了进一步研究的机会和挑战。