We introduce beta diffusion, a novel generative modeling method that integrates demasking and denoising to generate data within bounded ranges. Using scaled and shifted beta distributions, beta diffusion utilizes multiplicative transitions over time to create both forward and reverse diffusion processes, maintaining beta distributions in both the forward marginals and the reverse conditionals, given the data at any point in time. Unlike traditional diffusion-based generative models relying on additive Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence. We demonstrate that the proposed KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs, which can also be derived as the KLUBs of the same KL divergence with its two arguments swapped. The loss function of beta diffusion, expressed in terms of Bregman divergence, further supports the efficacy of KLUBs for optimization. Experimental results on both synthetic data and natural images demonstrate the unique capabilities of beta diffusion in generative modeling of range-bounded data and validate the effectiveness of KLUBs in optimizing diffusion models, thereby making them valuable additions to the family of diffusion-based generative models and the optimization techniques used to train them.

引入了β扩散的新型生成建模方法，结合去掩码和去噪的技术在有限范围内生成数据；将缩放和平移的贝塔分布与乘法转换结合使用，通过时间来创建正向和反向扩散过程，在任意时间点给定数据条件下维持正向边缘分布和反向条件分布；该方法优化KL散度上界，表明KL散度的两个参数互换时所得到的KL散度的负下界不如优化后的KL散度上界效果好；β扩散的损失函数以Bregman散度表示，进一步支持用KL散度上界进行优化；在合成数据和自然图像上的实验结果验证了β扩散在范围受限生成建模方面的独特能力，证实了KL散度上界在优化扩散模型方面的有效性，从而成为扩散式生成模型和训练它们所使用的优化技术中的有价值补充。

Beta扩散