As pretrained text-to-image diffusion models have become a useful tool for
image synthesis, people want to specify the results in various ways. In this
paper, we introduce a method to produce results with the same structure of a
target image but painted with colors from a reference image, i.e., appearance
transfer, especially following the semantic correspondence between the result
and the reference. E.g., the result wing takes color from the reference wing,
not the reference head. Existing methods rely on the query-key similarity
within self-attention layer, usually producing defective results. To this end,
we propose to find semantic correspondences and explicitly rearrange the
features according to the semantic correspondences. Extensive experiments show
the superiority of our method in various aspects: preserving the structure of
the target and reflecting the color from the reference according to the
semantic correspondences, even when the two images are not aligned.

介绍了一种通过语义对应关系对图像进行外观转移的方法，该方法使用预先训练的文本到图像扩散模型，并通过重新排列特征来提高结果的准确性。

以语义对应的扩散模型实现外貌转换

Eye-for-an-eye: Appearance Transfer with Semantic Correspondence in  Diffusion Models

As pretrained text-to-image diffusion models become increasingly powerful,
recent efforts have been made to distill knowledge from these text-to-image
pretrained models for optimizing a text-guided 3D model. Most of the existing
methods generate a holistic 3D model from a plain text input. This can be
problematic when the text describes a complex scene with multiple objects,
because the vectorized text embeddings are inherently unable to capture a
complex description with multiple entities and relationships. Holistic 3D
modeling of the entire scene further prevents accurate grounding of text
entities and concepts. To address this limitation, we propose GraphDreamer, a
novel framework to generate compositional 3D scenes from scene graphs, where
objects are represented as nodes and their interactions as edges. By exploiting
node and edge information in scene graphs, our method makes better use of the
pretrained text-to-image diffusion model and is able to fully disentangle
different objects without image-level supervision. To facilitate modeling of
object-wise relationships, we use signed distance fields as representation and
impose a constraint to avoid inter-penetration of objects. To avoid manual
scene graph creation, we design a text prompt for ChatGPT to generate scene
graphs based on text inputs. We conduct both qualitative and quantitative
experiments to validate the effectiveness of GraphDreamer in generating
high-fidelity compositional 3D scenes with disentangled object entities.

通过使用场景图生成组成 3D 场景的图形图像扩散预训练模型，利用节点和边的信息，GraphDreamer 能够生成高保真的分解对象实体的组合性 3D 场景。