The aim of this short note is to show that Denoising Diffusion Probabilistic Model DDPM, a non-homogeneous discrete-time Markov process, can be represented by a time-homogeneous continuous-time Markov process observed at non-uniformly sampled discrete times. Surprisingly, this continuous-time Markov process is the well-known and well-studied Ornstein-Ohlenbeck (OU) process, which was developed in 1930's for studying Brownian particles in Harmonic potentials. We establish the formal equivalence between DDPM and the OU process using its analytical solution. We further demonstrate that the design problem of the noise scheduler for non-homogeneous DDPM is equivalent to designing observation times for the OU process. We present several heuristic designs for observation times based on principled quantities such as auto-variance and Fisher Information and connect them to ad hoc noise schedules for DDPM. Interestingly, we show that the Fisher-Information-motivated schedule corresponds exactly the cosine schedule, which was developed without any theoretical foundation but is the current state-of-the-art noise schedule.

研究论文简述：本文表明，在观测到非均匀采样离散时间时，非均匀离散时间马尔可夫过程中的去噪扩散概率模型（DDPM）可以通过具有时间均匀性的连续时间马尔可夫过程来表示。我们使用其解析解建立了DDPM与已知的研究广泛的Ornstein-Ohlenbeck（OU）过程之间的形式上的等价关系。我们进一步证明了非均匀DDPM中的噪声调度器设计问题等价于OU过程的观测时间设计问题。我们基于如自动方差和费舍尔信息等原理性量化指标，提出了几种启发式的观测时间设计，并将它们与DDPM的特殊噪声调度相连接。有趣的是，我们展示了费舍尔信息驱动的调度与余弦调度完全一致，后者是没有任何理论基础但是目前最先进的噪声调度。

应用奥恩斯坦-乌伦贝克过程理解去噪扩散概率模型及其噪声计划