Recently normalizing flows have been gaining traction in text-to-speech (TTS)
and voice conversion (VC) due to their state-of-the-art (SOTA) performance.
Normalizing flows are unsupervised generative models. In this paper, we
introduce supervision to the training process of normalizing flows, without the
need for parallel data. We call this training paradigm AutoEncoder Normalizing
Flow (AE-Flow). It adds a reconstruction loss forcing the model to use
information from the conditioning to reconstruct an audio sample. Our goal is
to understand the impact of each component and find the right combination of
the negative log-likelihood (NLL) and the reconstruction loss in training
normalizing flows with coupling blocks. For that reason we will compare
flow-based mapping model trained with: (i) NLL loss, (ii) NLL and
reconstruction losses, as well as (iii) reconstruction loss only. Additionally,
we compare our model with SOTA VC baseline. The models are evaluated in terms
of naturalness, speaker similarity, intelligibility in many-to-many and
many-to-any VC settings. The results show that the proposed training paradigm
systematically improves speaker similarity and naturalness when compared to
regular training methods of normalizing flows. Furthermore, we show that our
method improves speaker similarity and intelligibility over the
state-of-the-art.

我们引入一种新的训练范式：自编码器归一化流（AE-Flow），它通过添加重构损失，使模型使用条件信息重构音频样本，结果表明该方法在正常化流的训练中系统地提高了说话者相似性和自然度。