The advancements in zero-shot text-to-speech (TTS) methods, based on
large-scale models, have demonstrated high fidelity in reproducing speaker
characteristics. However, these models are too large for practical daily use.
We propose a lightweight zero-shot TTS method using a mixture of adapters
(MoA). Our proposed method incorporates MoA modules into the decoder and the
variance adapter of a non-autoregressive TTS model. These modules enhance the
ability to adapt a wide variety of speakers in a zero-shot manner by selecting
appropriate adapters associated with speaker characteristics on the basis of
speaker embeddings. Our method achieves high-quality speech synthesis with
minimal additional parameters. Through objective and subjective evaluations, we
confirmed that our method achieves better performance than the baseline with
less than 40\% of parameters at 1.9 times faster inference speed. Audio samples
are available on our demo page
(this https URL).

基于大规模模型的零样本文本转语音（TTS）方法的进步展示了高保真度的说话者特征重现，但这些模型过于庞大以至于无法实际日常使用。我们提出了一种使用混合适配器（MoA）的轻量级零样本 TTS 方法。我们的方法将 MoA 模块整合到非自回归 TTS 模型的解码器和方差适配器中，通过根据说话者嵌入选择与说话者特征相关的适配器，以零样本方式增强了适应各种说话者的能力。我们的方法以最小的附加参数实现了高质量的语音合成。通过客观和主观评估，我们确认我们的方法在比基准少 40% 的参数下以 1.9 倍的推理速度实现了更好的性能。可以在我们的演示页面（此 https 网址）上找到音频样本。

轻量级零样本文本转语音与适配器混合模型

Lightweight Zero-shot Text-to-Speech with Mixture of Adapters

There are significant challenges for speaker adaptation in text-to-speech for
languages that are not widely spoken or for speakers with accents or dialects
that are not well-represented in the training data. To address this issue, we
propose the use of the "mixture of adapters" method. This approach involves
adding multiple adapters within a backbone-model layer to learn the unique
characteristics of different speakers. Our approach outperforms the baseline,
with a noticeable improvement of 5% observed in speaker preference tests when
using only one minute of data for each new speaker. Moreover, following the
adapter paradigm, we fine-tune only the adapter parameters (11% of the total
model parameters). This is a significant achievement in parameter-efficient
speaker adaptation, and one of the first models of its kind. Overall, our
proposed approach offers a promising solution to the speech synthesis
techniques, particularly for adapting to speakers from diverse backgrounds.

通过 “混合适配器” 方法，仅使用每个新说话者的一分钟数据，我们的方法在语音合成中提供了一种适应多样化背景说话者的有前途的解决方案，超过了基线，观察到 5％的说话者喜好测试的明显改善，并在适配器范例下只微调适配器参数（总模型参数的 11％），实现了参数高效的说话者适应，是其类别中的首个模型。