Deep generative models have achieved great success in unsupervised learning
with the ability to capture complex nonlinear relationships between latent
generating factors and observations. Among them, a factorized hierarchical
variational autoencoder (FHVAE) is a variational inference-based model that
formulates a hierarchical generative process for sequential data. Specifically,
an FHVAE model can learn disentangled and interpretable representations, which
have been proven useful for numerous speech applications, such as speaker
verification, robust speech recognition, and voice conversion. However, as we
will elaborate in this paper, the training algorithm proposed in the original
paper is not scalable to datasets of thousands of hours, which makes this model
less applicable on a larger scale. After identifying limitations in terms of
runtime, memory, and hyperparameter optimization, we propose a hierarchical
sampling training algorithm to address all three issues. Our proposed method is
evaluated comprehensively on a wide variety of datasets, ranging from 3 to
1,000 hours and involving different types of generating factors, such as
recording conditions and noise types. In addition, we also present a new
visualization method for qualitatively evaluating the performance with respect
to the interpretability and disentanglement. Models trained with our proposed
algorithm demonstrate the desired characteristics on all the datasets.

本文介绍一种基于变分推断的分层生成过程的因式分解分层变分自编码器（FHVAE）模型，该模型在许多语音应用方面已被证明可行，但其原始论文中提出的训练算法不适用于规模更大的数据集。本文提出了一种分层采样训练算法来解决这一问题，并对不同类型的数据集进行了全面评估，证明了我们提出的算法对所有数据集都表现出所需的特性。

可扩展的分解层次变分自编码器训练

Scalable Factorized Hierarchical Variational Autoencoder Training

We present a factorized hierarchical variational autoencoder, which learns
disentangled and interpretable representations from sequential data without
supervision. Specifically, we exploit the multi-scale nature of information in
sequential data by formulating it explicitly within a factorized hierarchical
graphical model that imposes sequence-dependent priors and sequence-independent
priors to different sets of latent variables. The model is evaluated on two
speech corpora to demonstrate, qualitatively, its ability to transform speakers
or linguistic content by manipulating different sets of latent variables; and
quantitatively, its ability to outperform an i-vector baseline for speaker
verification and reduce the word error rate by as much as 35% in mismatched
train/test scenarios for automatic speech recognition tasks.

本文提出一种分解的分层变分自动编码器，能够在无监督的情况下从序列数据中学习分离和可解释的表征。该模型基于多尺度信息的分层结构，实现了针对不同潜变量集合的序列依赖先验和序列无关先验。通过在两个语音语料库上的实验，表明该模型能够通过操纵不同的潜变量集合来转换说话人或语言内容，并在说话人验证和自动语音识别任务中优于基线模型。