State-of-the-art Variational Auto-Encoders (VAEs) for learning disentangled
latent representations give impressive results in discovering features like
pitch, pause duration, and accent in speech data, leading to highly
controllable text-to-speech (TTS) synthesis. However, these LSTM-based VAEs
fail to learn latent clusters of speaker attributes when trained on either
limited or noisy datasets. Further, different latent variables start encoding
the same features, limiting the control and expressiveness during speech
synthesis. To resolve these issues, we propose RTI-VAE (Reordered Transformer
with Information reduction VAE) where we minimize the mutual information
between different latent variables and devise a modified Transformer
architecture with layer reordering to learn controllable latent representations
in speech data. We show that RTI-VAE reduces the cluster overlap of speaker
attributes by at least 30\% over LSTM-VAE and by at least 7\% over vanilla
Transformer-VAE.

提出了一种 RTI-VAE 方法，使用修改过的 Transformer 架构和信息减少技术来学习可控制的语音数据的潜在变量，从而降低说话者属性聚类的重叠率，相比于 LSTM-VAE 和 vanilla Transformer-VAE，降低了至少 30％和至少 7％的重叠率。