Discrete diffusion models have seen a surge of attention with applications on
naturally discrete data such as language and graphs. Although discrete-time
discrete diffusion has been established for a while, only recently Campbell et
al. (2022) introduced the first framework for continuous-time discrete
diffusion. However, their training and sampling processes differ significantly
from the discrete-time version, necessitating nontrivial approximations for
tractability. In this paper, we first present a series of mathematical
simplifications of the variational lower bound that enable more accurate and
easy-to-optimize training for discrete diffusion. In addition, we derive a
simple formulation for backward denoising that enables exact and accelerated
sampling, and importantly, an elegant unification of discrete-time and
continuous-time discrete diffusion. Thanks to simpler analytical formulations,
both forward and now also backward probabilities can flexibly accommodate any
noise distribution, including different noise distributions for multi-element
objects. Experiments show that our proposed USD3 (for Unified Simplified
Discrete Denoising Diffusion) outperform all SOTA baselines on established
datasets. We open-source our unified code at
this https URL

提出了一种用于简化离散扩散的数学简化方案，同时还提出了一种能够精确和加速采样的简单公式，并通过创建一个统一的模型，简化离散扩散的前向和后向概率计算，取得了在现有数据集上优于其他方法的效果。

改善和统一的离散和连续时间离散去噪扩散

Improving and Unifying Discrete&Continuous-time Discrete Denoising  Diffusion

Because diffusion models have shown impressive performances in a number of
tasks, such as image synthesis, there is a trend in recent works to prove (with
certain assumptions) that these models have strong approximation capabilities.
In this paper, we show that current diffusion models actually have an
expressive bottleneck in backward denoising and some assumption made by
existing theoretical guarantees is too strong. Based on this finding, we prove
that diffusion models have unbounded errors in both local denoising and global
approximation. In light of our theoretical studies, we introduce soft mixture
denoising (SMD), an expressive and efficient model for backward denoising. SMD
not only permits diffusion models to well approximate any Gaussian mixture
distributions in theory, but also is simple and efficient for implementation.
Our experiments on multiple image datasets show that SMD significantly improves
different types of diffusion models (e.g., DDPM), especially in the situation
of few backward iterations.

扩散模型在图像合成等任务中表现出色，但是我们发现当前的扩散模型在反向去噪和现有理论保证过程中存在表达瓶颈和过强的假设，导致局部去噪和全局逼近存在无界误差。基于我们的理论研究，我们提出了一种简单高效的反向去噪模型 —— 软混合去噪（SMD），它不仅在理论上允许扩散模型良好逼近任意高斯混合分布，而且在实现上具备简单高效的特点。我们在多个图像数据集上的实验证明，SMD 显著改进了各种类型的扩散模型（例如 DDPM），特别是在少量反向迭代的情况下。