Score function estimation is the cornerstone of both training and sampling from diffusion generative models. Despite this fact, the most commonly used estimators are either biased neural network approximations or high variance Monte Carlo estimators based on the conditional score. We introduce a novel nearest neighbour score function estimator which utilizes multiple samples from the training set to dramatically decrease estimator variance. We leverage our low variance estimator in two compelling applications. Training consistency models with our estimator, we report a significant increase in both convergence speed and sample quality. In diffusion models, we show that our estimator can replace a learned network for probability-flow ODE integration, opening promising new avenues of future research.

评分函数估计是训练和采样扩散生成模型的基石。我们引入了一种利用训练集中的多个样本来大幅降低估计方差的新型最近邻评分函数估计器，并将其应用于一致性模型训练中，加快收敛速度并提高样本质量。在扩散模型中，我们展示了该估计器可以替代学习网络用于概率流ODE积分，为未来研究开辟了有希望的新途径。

扩散生成模型的最近邻评分估计器