Generative models in vision have seen rapid progress due to algorithmic
improvements and the availability of high-quality image datasets. In this
paper, we offer contributions in both these areas to enable similar progress in
audio modeling. First, we detail a powerful new WaveNet-style autoencoder model
that conditions an autoregressive decoder on temporal codes learned from the
raw audio waveform. Second, we introduce NSynth, a large-scale and high-quality
dataset of musical notes that is an order of magnitude larger than comparable
public datasets. Using NSynth, we demonstrate improved qualitative and
quantitative performance of the WaveNet autoencoder over a well-tuned spectral
autoencoder baseline. Finally, we show that the model learns a manifold of
embeddings that allows for morphing between instruments, meaningfully
interpolating in timbre to create new types of sounds that are realistic and
expressive.

本研究提出了一种基于 WaveNet 风格自编码器的新型音频模型，进一步介绍了一个规模较大质量较高的音乐数据集 NSynth。使用 NSynth 数据集，我们证明 WaveNet 自编码器比使用频谱自编码器基线的性能有显著提高，同时演示了该模型学习嵌入的能力，实现了在音色上的差值并创造了新类型的逼真和有表现力的声音。