Deep generative models can generate high-fidelity audio conditioned on
various types of representations (e.g., mel-spectrograms, Mel-frequency
Cepstral Coefficients (MFCC)). Recently, such models have been used to
synthesize audio waveforms conditioned on highly compressed representations.
Although such methods produce impressive results, they are prone to generate
audible artifacts when the conditioning is flawed or imperfect. An alternative
modeling approach is to use diffusion models. However, these have mainly been
used as speech vocoders (i.e., conditioned on mel-spectrograms) or generating
relatively low sampling rate signals. In this work, we propose a high-fidelity
multi-band diffusion-based framework that generates any type of audio modality
(e.g., speech, music, environmental sounds) from low-bitrate discrete
representations. At equal bit rate, the proposed approach outperforms
state-of-the-art generative techniques in terms of perceptual quality. Training
and, evaluation code, along with audio samples, are available on the
facebookresearch/audiocraft Github page.

利用高保真多带扩散模型，基于低比特率离散表示生成任何类型的音频，其感知质量比最先进的生成技术更好。

从离散的标记到高保真音频的多频带扩散

From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion

Several recent work on speech synthesis have employed generative adversarial
networks (GANs) to produce raw waveforms. Although such methods improve the
sampling efficiency and memory usage, their sample quality has not yet reached
that of autoregressive and flow-based generative models. In this work, we
propose HiFi-GAN, which achieves both efficient and high-fidelity speech
synthesis. As speech audio consists of sinusoidal signals with various periods,
we demonstrate that modeling periodic patterns of an audio is crucial for
enhancing sample quality. A subjective human evaluation (mean opinion score,
MOS) of a single speaker dataset indicates that our proposed method
demonstrates similarity to human quality while generating 22.05 kHz
high-fidelity audio 167.9 times faster than real-time on a single V100 GPU. We
further show the generality of HiFi-GAN to the mel-spectrogram inversion of
unseen speakers and end-to-end speech synthesis. Finally, a small footprint
version of HiFi-GAN generates samples 13.4 times faster than real-time on CPU
with comparable quality to an autoregressive counterpart.

该研究提出了 HiFi-GAN，一种高效且高保真度的语音合成方法，通过对声音的周期性模式进行建模，生成的音频质量接近于人类水平。