Stochastic latent variable models (LVMs) achieve state-of-the-art performance
on natural image generation but are still inferior to deterministic models on
speech. In this paper, we develop a speech benchmark of popular temporal LVMs
and compare them against state-of-the-art deterministic models. We report the
likelihood, which is a much used metric in the image domain, but rarely, or
incomparably, reported for speech models. To assess the quality of the learned
representations, we also compare their usefulness for phoneme recognition.
Finally, we adapt the Clockwork VAE, a state-of-the-art temporal LVM for video
generation, to the speech domain. Despite being autoregressive only in latent
space, we find that the Clockwork VAE can outperform previous LVMs and reduce
the gap to deterministic models by using a hierarchy of latent variables.

该研究发展了一种基于时间的语音 LVM 基准，并将其与实用模型进行了比较，结合使用度量标准以及潜在变量等进行定量的分析。同时，也探究了建立在多个潜在变量层次上的时钟 VAE 模型在提高语音生成表现中的应用。