Recent advancements in neural vocoding are predominantly driven by Generative
Adversarial Networks (GANs) operating in the time-domain. While effective, this
approach neglects the inductive bias offered by time-frequency representations,
resulting in reduntant and computionally-intensive upsampling operations.
Fourier-based time-frequency representation is an appealing alternative,
aligning more accurately with human auditory perception, and benefitting from
well-established fast algorithms for its computation. Nevertheless, direct
reconstruction of complex-valued spectrograms has been historically
problematic, primarily due to phase recovery issues. This study seeks to close
this gap by presenting Vocos, a new model that addresses the key challenges of
modeling spectral coefficients. Vocos demonstrates improved computational
efficiency, achieving an order of magnitude increase in speed compared to
prevailing time-domain neural vocoding approaches. As shown by objective
evaluation, Vocos not only matches state-of-the-art audio quality, but thanks
to frequency-aware generator, also effectively mitigates the periodicity issues
frequently associated with time-domain GANs. The source code and model weights
have been open-sourced at this https URL

本研究提出了 Vocos 模型，该模型利用基于傅里叶变换的时频表示解决了复杂值谱图的直接重建问题，并显著提高了计算效率，具有与时域神经编码方法相媲美的音频质量和具有频率感知能力的生成器。