Recent advancements in speech synthesis have leveraged GAN-based networks
like HiFi-GAN and BigVGAN to produce high-fidelity waveforms from
mel-spectrograms. However, these networks are computationally expensive and
parameter-heavy. iSTFTNet addresses these limitations by integrating inverse
short-time Fourier transform (iSTFT) into the network, achieving both speed and
parameter efficiency. In this paper, we introduce an extension to iSTFTNet,
termed HiFTNet, which incorporates a harmonic-plus-noise source filter in the
time-frequency domain that uses a sinusoidal source from the fundamental
frequency (F0) inferred via a pre-trained F0 estimation network for fast
inference speed. Subjective evaluations on LJSpeech show that our model
significantly outperforms both iSTFTNet and HiFi-GAN, achieving
ground-truth-level performance. HiFTNet also outperforms BigVGAN-base on
LibriTTS for unseen speakers and achieves comparable performance to BigVGAN
while being four times faster with only $1/6$ of the parameters. Our work sets
a new benchmark for efficient, high-quality neural vocoding, paving the way for
real-time applications that demand high quality speech synthesis.

本文介绍了一种名为 HiFTNet 的扩展模型，通过在时频域中使用基频（F0）的正弦源来快速推理，结合了反短时傅里叶变换（iSTFT）和谐波加噪声源滤波器，实现了高效、高质量的神经音频编码，为实时高质量语音合成应用开辟了新的前景。