Given a factorization of an image into a sum of linear components, we present
a zero-shot method to control each individual component through diffusion model
sampling. For example, we can decompose an image into low and high spatial
frequencies and condition these components on different text prompts. This
produces hybrid images, which change appearance depending on viewing distance.
By decomposing an image into three frequency subbands, we can generate hybrid
images with three prompts. We also use a decomposition into grayscale and color
components to produce images whose appearance changes when they are viewed in
grayscale, a phenomena that naturally occurs under dim lighting. And we explore
a decomposition by a motion blur kernel, which produces images that change
appearance under motion blurring. Our method works by denoising with a
composite noise estimate, built from the components of noise estimates
conditioned on different prompts. We also show that for certain decompositions,
our method recovers prior approaches to compositional generation and spatial
control. Finally, we show that we can extend our approach to generate hybrid
images from real images. We do this by holding one component fixed and
generating the remaining components, effectively solving an inverse problem.

通过将图像因子分解为线性组件之和，我们提出了一种零样本方法来通过扩散模型采样来控制每个单独的组件。我们通过分解图像为低频和高频空间成分并基于不同的文本提示来调整这些成分，可以产生根据观察距离而改变外观的混合图像。我们还使用将图像分解为灰度和彩色成分，以生成在灰度下外观改变的图像，这在昏暗的光照下自然发生。此外，我们还通过运动模糊核将图像分解，从而生成在运动模糊下外观改变的图像。我们的方法通过使用组合噪声估计进行去噪，该估计是通过在不同的提示条件下对噪声估计的成分进行构建而得到的。我们还展示出，在某些分解情况下，我们的方法可以恢复先前的生成和空间控制方法。最后，我们展示了我们可以将该方法扩展到从真实图像生成混合图像。我们通过固定一个组件并生成其余的组件来解决一个逆问题。

因子化扩散：通过噪声分解引起的感知错觉

Factorized Diffusion: Perceptual Illusions by Noise Decomposition

Hybrid images is a technique to generate images with two interpretations that
change as a function of viewing distance. It has been utilized to study
multiscale processing of images by the human visual system. Using 63,000 hybrid
images across 10 fruit categories, here we show that predictions of deep
learning vision models qualitatively matches with the human perception of these
images. Our results provide yet another evidence in support of the hypothesis
that Convolutional Neural Networks (CNNs) and Transformers are good at modeling
the feedforward sweep of information in the ventral stream of visual cortex.
Code and data is available at this https URL.

混合图像技术（hybrid images）与深度学习视觉模型在研究人类视觉系统的多尺度图像处理方面具有定性一致性且卷积神经网络（CNN）和 Transformer 在视觉皮层腹侧通路中的前向信息传递建模方面表现优异。