We consider the problem of independently, in a disentangled fashion, controlling the outputs of text-to-image diffusion models with color and style attributes of a user-supplied reference image. We present the first training-free, test-time-only method to disentangle and condition text-to-image models on color and style attributes from reference image. To realize this, we propose two key innovations. Our first contribution is to transform the latent codes at inference time using feature transformations that make the covariance matrix of current generation follow that of the reference image, helping meaningfully transfer color. Next, we observe that there exists a natural disentanglement between color and style in the LAB image space, which we exploit to transform the self-attention feature maps of the image being generated with respect to those of the reference computed from its L channel. Both these operations happen purely at test time and can be done independently or merged. This results in a flexible method where color and style information can come from the same reference image or two different sources, and a new generation can seamlessly fuse them in either scenario.

该研究解决了如何以解耦方式独立控制文本到图像生成模型输出的颜色和风格属性问题。提出了一种首次不依赖训练且仅在测试时进行的方法，通过特征变换和LAB图像空间中的自然分离，灵活地将用户提供的参考图像的颜色和风格应用于生成图像，从而实现了有效的特征融合。

无训练的约束文本到图像合成的颜色风格分离