For our submission to the ZeroSpeech 2019 challenge, we apply discrete
latent-variable neural networks to unlabelled speech and use the discovered
units for speech synthesis. Unsupervised discrete subword modelling could be
useful for studies of phonetic category learning in infants or in low-resource
speech technology requiring symbolic input. We use an autoencoder (AE)
architecture with intermediate discretisation. We decouple acoustic unit
discovery from speaker modelling by conditioning the AE's decoder on the
training speaker identity. At test time, unit discovery is performed on speech
from an unseen speaker, followed by unit decoding conditioned on a known target
speaker to obtain reconstructed filterbanks. This output is fed to a neural
vocoder to synthesise speech in the target speaker's voice. For discretisation,
categorical variational autoencoders (CatVAEs), vector-quantised VAEs (VQ-VAEs)
and straight-through estimation are compared at different compression levels on
two languages. Our final model uses convolutional encoding, VQ-VAE
discretisation, deconvolutional decoding and an FFTNet vocoder. We show that
decoupled speaker conditioning intrinsically improves discrete acoustic
representations, yielding competitive synthesis quality compared to the
challenge baseline.

本文介绍了使用离散潜变量神经网络对无标签语音进行离散子单词建模，使用自编码器架构进行中间离散化，在训练中，根据模型的特点对语音单元重新建模，以甄别可离散化的语音特征。在测试中，离线甄别采用未知说话者的语音，通过已知目标说话者的条件下的解码来获得重构的滤波器组。最后，使用神经声码器将输出进行合成，比较分别使用分类变分自编码器（CatVAEs）, 矢量量化 VAEs（VQ-VAEs）和直通估计在两种语言上的不同压缩水平，并发现该模型可以在离散表示方面产生竞争性的合成质量