Denoising Diffusion Probabilistic Models have shown extraordinary ability on
various generative tasks. However, their slow inference speed renders them
impractical in speech synthesis. This paper proposes a linear diffusion model
(LinDiff) based on an ordinary differential equation to simultaneously reach
fast inference and high sample quality. Firstly, we employ linear interpolation
between the target and noise to design a diffusion sequence for training, while
previously the diffusion path that links the noise and target is a curved
segment. When decreasing the number of sampling steps (i.e., the number of line
segments used to fit the path), the ease of fitting straight lines compared to
curves allows us to generate higher quality samples from a random noise with
fewer iterations. Secondly, to reduce computational complexity and achieve
effective global modeling of noisy speech, LinDiff employs a patch-based
processing approach that partitions the input signal into small patches. The
patch-wise token leverages Transformer architecture for effective modeling of
global information. Adversarial training is used to further improve the sample
quality with decreased sampling steps. We test proposed method with speech
synthesis conditioned on acoustic feature (Mel-spectrograms). Experimental
results verify that our model can synthesize high-quality speech even with only
one diffusion step. Both subjective and objective evaluations demonstrate that
our model can synthesize speech of a quality comparable to that of
autoregressive models with faster synthesis speed (3 diffusion steps).

本文提出了一种基于普通微分方程的线性扩散模型（LinDiff），其旨在实现快速推理和高采样质量，并通过基于补丁的处理方法来减少计算复杂性和实现嘈杂语音的有效全局建模，并使用对抗性训练进一步改善样本质量，并在 Mel - 声谱图上条件语音合成中进行测试，实验结果表明，即使只有一个扩散步骤，该模型也可以合成高质量的语音，而且在更快的合成速度下合成质量与自回归模型相媲美。