Neural vocoders model the raw audio waveform and synthesize high-quality
audio, but even the highly efficient ones, like MB-MelGAN and LPCNet, fail to
run real-time on a low-end device like a smartglass. A pure digital signal
processing (DSP) based vocoder can be implemented via lightweight fast Fourier
transforms (FFT), and therefore, is a magnitude faster than any neural vocoder.
A DSP vocoder often gets a lower audio quality due to consuming over-smoothed
acoustic model predictions of approximate representations for the vocal tract.
In this paper, we propose an ultra-lightweight differential DSP (DDSP) vocoder
that uses a jointly optimized acoustic model with a DSP vocoder, and learns
without an extracted spectral feature for the vocal tract. The model achieves
audio quality comparable to neural vocoders with a high average MOS of 4.36
while being efficient as a DSP vocoder. Our C++ implementation, without any
hardware-specific optimization, is at 15 MFLOPS, surpasses MB-MelGAN by 340
times in terms of FLOPS, and achieves a vocoder-only RTF of 0.003 and overall
RTF of 0.044 while running single-threaded on a 2GHz Intel Xeon CPU.

提出了一种超轻量级的差分 DSP (DDSP) vocoder，该 vocoder 使用了一个经过联合优化的声学模型和 DSP vocoder，学习过程中不需要提取声道的光谱特征，实现了与神经 vocoder 相当的音频质量，同时作为一个 DSP vocoder 非常高效。在 2GHz 的 Intel Xeon CPU 上单线程运行时，无需任何硬件特定优化，C++ 实现的 FLOPS 达到 15 MFLOPS，相对于 MB-MelGAN 的 FLOPS，高出 340 倍，仅需 0.003 的 vocoder-only RTF 和 0.044 的总 RTF。