We present a novel method for training score-based generative models which
uses nonlinear noising dynamics to improve learning of structured
distributions. Generalizing to a nonlinear drift allows for additional
structure to be incorporated into the dynamics, thus making the training better
adapted to the data, e.g., in the case of multimodality or (approximate)
symmetries. Such structure can be obtained from the data by an inexpensive
preprocessing step. The nonlinear dynamics introduces new challenges into
training which we address in two ways: 1) we develop a new nonlinear denoising
score matching (NDSM) method, 2) we introduce neural control variates in order
to reduce the variance of the NDSM training objective. We demonstrate the
effectiveness of this method on several examples: a) a collection of
low-dimensional examples, motivated by clustering in latent space, b)
high-dimensional images, addressing issues with mode collapse, small training
sets, and approximate symmetries, the latter being a challenge for methods
based on equivariant neural networks, which require exact symmetries.

提出了一种新颖的训练基于分数的生成模型的方法，通过非线性加噪动力学改进结构化分布的学习，从而使其更好地适应数据，在数据预处理步骤中获取结构并引入了两种方法解决非线性动力学训练带来的挑战，展示了该方法在几个示例上的有效性。