In this work, we propose a novel method for modeling numerous speakers, which
enables expressing the overall characteristics of speakers in detail like a
trained multi-speaker model without additional training on the target speaker's
dataset. Although various works with similar purposes have been actively
studied, their performance has not yet reached that of trained multi-speaker
models due to their fundamental limitations. To overcome previous limitations,
we propose effective methods for feature learning and representing target
speakers' speech characteristics by discretizing the features and conditioning
them to a speech synthesis model. Our method obtained a significantly higher
similarity mean opinion score (SMOS) in subjective similarity evaluation than
seen speakers of a best-performing multi-speaker model, even with unseen
speakers. The proposed method also outperforms a zero-shot method by
significant margins. Furthermore, our method shows remarkable performance in
generating new artificial speakers. In addition, we demonstrate that the
encoded latent features are sufficiently informative to reconstruct an original
speaker's speech completely. It implies that our method can be used as a
general methodology to encode and reconstruct speakers' characteristics in
various tasks.

提出了一种用于建模众多发言人的新方法，并通过对特征进行离散化和将其与语音合成模型相结合来表示目标发言人的语音特征。该方法在主观相似性评估中获得了较高的相似度平均意见分数（SMOS），甚至对于未见过的发言人，其性能优于最佳多发言人模型的已见发言人，而且也显著优于零样本方法。此外，该方法在生成新的虚拟发言人方面表现出色，并且通过编码潜在特征能够完全重构原始发言人的语音，说明该方法可以作为在各种任务中对发言人特征进行编码和重构的通用方法。