How do diffusion generative models convert pure noise into meaningful images?
We argue that generation involves first committing to an outline, and then to
finer and finer details. The corresponding reverse diffusion process can be
modeled by dynamics on a (time-dependent) high-dimensional landscape full of
Gaussian-like modes, which makes the following predictions: (i) individual
trajectories tend to be very low-dimensional; (ii) scene elements that vary
more within training data tend to emerge earlier; and (iii) early perturbations
substantially change image content more often than late perturbations. We show
that the behavior of a variety of trained unconditional and conditional
diffusion models like Stable Diffusion is consistent with these predictions.
Finally, we use our theory to search for the latent image manifold of diffusion
models, and propose a new way to generate interpretable image variations. Our
viewpoint suggests generation by GANs and diffusion models have unexpected
similarities.

本文针对扩散生成模型如何将纯噪声转化为有意义的图像进行了研究，提出了基于高维景观的动力学建模方法，揭示了不同训练数据中的场景元素所呈现的具体细节先后顺序，并提出了一种生成可解释图像变化的新方法。

扩散模型生成如画家一样的图像：线条先行，细节随后的分析理论

Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later

Semantic image inpainting is a challenging task where large missing regions
have to be filled based on the available visual data. Existing methods which
extract information from only a single image generally produce unsatisfactory
results due to the lack of high level context. In this paper, we propose a
novel method for semantic image inpainting, which generates the missing content
by conditioning on the available data. Given a trained generative model, we
search for the closest encoding of the corrupted image in the latent image
manifold using our context and prior losses. This encoding is then passed
through the generative model to infer the missing content. In our method,
inference is possible irrespective of how the missing content is structured,
while the state-of-the-art learning based method requires specific information
about the holes in the training phase. Experiments on three datasets show that
our method successfully predicts information in large missing regions and
achieves pixel-level photorealism, significantly outperforming the
state-of-the-art methods.

本文提出了一种基于生成模型的新型语义图像修复方法，该方法可在缺失的内容相互独立的情况下进行推理，并取得超越其他方法的像素级真实感。