Recent advancements in speech synthesis have leveraged GAN-based networks
like HiFi-GAN and BigVGAN to produce high-fidelity waveforms from
mel-spectrograms. However, these networks are computationally expensive and
parameter-heavy. iSTFTNet addresses these limitations by integrating inverse
short-time Fourier transform (iSTFT) into the network, achieving both speed and
parameter efficiency. In this paper, we introduce an extension to iSTFTNet,
termed HiFTNet, which incorporates a harmonic-plus-noise source filter in the
time-frequency domain that uses a sinusoidal source from the fundamental
frequency (F0) inferred via a pre-trained F0 estimation network for fast
inference speed. Subjective evaluations on LJSpeech show that our model
significantly outperforms both iSTFTNet and HiFi-GAN, achieving
ground-truth-level performance. HiFTNet also outperforms BigVGAN-base on
LibriTTS for unseen speakers and achieves comparable performance to BigVGAN
while being four times faster with only $1/6$ of the parameters. Our work sets
a new benchmark for efficient, high-quality neural vocoding, paving the way for
real-time applications that demand high quality speech synthesis.

本文介绍了一种名为 HiFTNet 的扩展模型，通过在时频域中使用基频（F0）的正弦源来快速推理，结合了反短时傅里叶变换（iSTFT）和谐波加噪声源滤波器，实现了高效、高质量的神经音频编码，为实时高质量语音合成应用开辟了新的前景。

HiFTNet：具备谐波加噪滤波器和逆短时傅里叶变换的高速高质量神经声码器

HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise  Filter and Inverse Short Time Fourier Transform

Recent advancements in neural vocoding are predominantly driven by Generative
Adversarial Networks (GANs) operating in the time-domain. While effective, this
approach neglects the inductive bias offered by time-frequency representations,
resulting in reduntant and computionally-intensive upsampling operations.
Fourier-based time-frequency representation is an appealing alternative,
aligning more accurately with human auditory perception, and benefitting from
well-established fast algorithms for its computation. Nevertheless, direct
reconstruction of complex-valued spectrograms has been historically
problematic, primarily due to phase recovery issues. This study seeks to close
this gap by presenting Vocos, a new model that addresses the key challenges of
modeling spectral coefficients. Vocos demonstrates improved computational
efficiency, achieving an order of magnitude increase in speed compared to
prevailing time-domain neural vocoding approaches. As shown by objective
evaluation, Vocos not only matches state-of-the-art audio quality, but thanks
to frequency-aware generator, also effectively mitigates the periodicity issues
frequently associated with time-domain GANs. The source code and model weights
have been open-sourced at this https URL

本研究提出了 Vocos 模型，该模型利用基于傅里叶变换的时频表示解决了复杂值谱图的直接重建问题，并显著提高了计算效率，具有与时域神经编码方法相媲美的音频质量和具有频率感知能力的生成器。

Vocos：面向高质量音频合成的时域和傅里叶神经声码器之间的差距的缩小

Vocos: Closing the gap between time-domain and Fourier-based neural  vocoders for high-quality audio synthesis

This paper introduces UnDiff, a diffusion probabilistic model capable of
solving various speech inverse tasks. Being once trained for speech waveform
generation in an unconditional manner, it can be adapted to different tasks
including degradation inversion, neural vocoding, and source separation. In
this paper, we, first, tackle the challenging problem of unconditional waveform
generation by comparing different neural architectures and preconditioning
domains. After that, we demonstrate how the trained unconditional diffusion
could be adapted to different tasks of speech processing by the means of recent
developments in post-training conditioning of diffusion models. Finally, we
demonstrate the performance of the proposed technique on the tasks of bandwidth
extension, declipping, vocoding, and speech source separation and compare it to
the baselines. The codes will be released soon.

本文介绍了 UnDiff，一种扩散概率模型，能够解决各种语音反问题，并可以应用于条件不同的任务，如降级反演、神经声波编解码和语音来源分离等。首先，我们比较了不同的神经架构和预处理方法来解决无条件波形生成的难题，并通过最新的扩散模型后处理技术演示了如何将无条件扩散模型应用于语音处理中的不同任务。最终，在带宽扩展、去剪辑、声学编解码和语音来源分离等任务上展示了所提出的技术的性能，并与基线进行比较。