We introduce a new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching. Because gradients might be ill-defined when the data resides on low-dimensional manifolds, we perturb the data with different levels of Gaussian noise and jointly estimate the corresponding scores, i.e., the vector fields of gradients of the perturbed data distribution for all noise levels. For sampling, we propose an annealed Langevin dynamics where we use gradients corresponding to gradually decreasing noise levels as the sampling process gets closer to the data manifold. Our framework allows flexible model architectures, requires no sampling during training or the use of adversarial methods, and provides a learning objective that can be used for principled model comparisons. Our models produce samples comparable to GANs on MNIST, CelebA and CIFAR-10 datasets, achieving a new state-of-the-art inception score of 8.91 on CIFAR-10. Additionally, we demonstrate that our models learn effective representations via image inpainting experiments.

本文介绍了一种新的生成模型，利用评分匹配来估计数据分布的梯度，通过Langevin动力学生成样本。我们的框架使得模型架构更加灵活，无需在训练期间进行抽样或使用对抗性方法，提供了可用于基于原则的模型比较的学习目标。在MNIST、CelebA和CIFAR-10数据集上，我们的模型产生的样本与GAN相当，实现了CIFAR-10 inception得分的新的最先进水平为8.87。此外，我们通过图像修补实验证明了我们的模型学习到了有效的表示。

通过估计数据分布梯度的方式进行生成建模