We present a method for semantically transferring the visual appearance of
one natural image to another. Specifically, our goal is to generate an image in
which objects in a source structure image are "painted" with the visual
appearance of their semantically related objects in a target appearance image.
Our method works by training a generator given only a single
structure/appearance image pair as input. To integrate semantic information
into our framework - a pivotal component in tackling this task - our key idea
is to leverage a pre-trained and fixed Vision Transformer (ViT) model which
serves as an external semantic prior. Specifically, we derive novel
representations of structure and appearance extracted from deep ViT features,
untwisting them from the learned self-attention modules. We then establish an
objective function that splices the desired structure and appearance
representations, interweaving them together in the space of ViT features. Our
framework, which we term "Splice", does not involve adversarial training, nor
does it require any additional input information such as semantic segmentation
or correspondences, and can generate high-resolution results, e.g., work in HD.
We demonstrate high quality results on a variety of in-the-wild image pairs,
under significant variations in the number of objects, their pose and
appearance.

本文介绍了一种名为 Splice 的方法，通过集成 ViT 模型的语义信息，将一个自然图像中的物体的视觉外观转移到另一个图像中，该方法不需要使用对抗训练，也不需要任何额外的输入信息。

利用 ViT 特征进行语义外观转换

Splicing ViT Features for Semantic Appearance Transfer

This work aims at transferring a Generative Adversarial Network (GAN)
pre-trained on one image domain to a new domain referring to as few as just one
target image. The main challenge is that, under limited supervision, it is
extremely difficult to synthesize photo-realistic and highly diverse images,
while acquiring representative characters of the target. Different from
existing approaches that adopt the vanilla fine-tuning strategy, we import two
lightweight modules to the generator and the discriminator respectively.
Concretely, we introduce an attribute adaptor into the generator yet freeze its
original parameters, through which it can reuse the prior knowledge to the most
extent and hence maintain the synthesis quality and diversity. We then equip
the well-learned discriminator backbone with an attribute classifier to ensure
that the generator captures the appropriate characters from the reference.
Furthermore, considering the poor diversity of the training data (i.e., as few
as only one image), we propose to also constrain the diversity of the
generative domain in the training process, alleviating the optimization
difficulty. Our approach brings appealing results under various settings,
substantially surpassing state-of-the-art alternatives, especially in terms of
synthesis diversity. Noticeably, our method works well even with large domain
gaps, and robustly converges within a few minutes for each experiment.

研究如何转移使用 GAN 训练的一个图像领域到一个新的领域，只需一个目标图像；通过使用属性适配器和属性分类器的轻量级模块，对生成器和鉴别器进行改进，从而提高合成质量和多样性，并在训练过程中约束生成领域的多样性，不仅在各种设置下带来了具有吸引力的结果，而且能够在几分钟内提供稳健的收敛。

单次生成对抗网络域自适应

One-Shot Generative Domain Adaptation

Given a pair of images-target person and garment on another person-we
automatically generate the target person in the given garment. Previous methods
mostly focused on texture transfer via paired data training, while overlooking
body shape deformations, skin color, and seamless blending of garment with the
person. This work focuses on those three components, while also not requiring
paired data training. We designed a pose conditioned StyleGAN2 architecture
with a clothing segmentation branch that is trained on images of people wearing
garments. Once trained, we propose a new layered latent space interpolation
method that allows us to preserve and synthesize skin color and target body
shape while transferring the garment from a different person. We demonstrate
results on high resolution 512x512 images, and extensively compare to state of
the art in try-on on both latent space generated and real images.

本文提出了一种使用姿势条件的 StyleGAN2 神经网络架构和服装分割分支来实现图像转移的方法，该方法可以保存和合成皮肤颜色和目标人物的体形，同时转移来自不同人物的衣服，从而完成了穿衣试衣的需求。