Recent advances in style transfer text-to-speech (TTS) have improved the
expressiveness of synthesized speech. Despite these advancements, encoding
stylistic information from diverse and unseen reference speech remains
challenging. This paper introduces StyleMoE, an approach that divides the
embedding space, modeled by the style encoder, into tractable subsets handled
by style experts. The proposed method replaces the style encoder in a TTS
system with a Mixture of Experts (MoE) layer. By utilizing a gating network to
route reference speeches to different style experts, each expert specializes in
aspects of the style space during optimization. Our experiments objectively and
subjectively demonstrate the effectiveness of our proposed method in increasing
the coverage of the style space for diverse and unseen styles. This approach
can enhance the performance of existing state-of-the-art style transfer TTS
models, marking the first study of MoE in style transfer TTS to our knowledge.

该研究提出了 StyleMoE 方法，通过将风格编码器建模的嵌入空间划分为可处理的子集，然后由专家处理。实验结果明确展示了该方法在增加多样且未知风格的风格空间覆盖方面的有效性，能够提升现有领先的风格转换语音合成模型的性能，并首次将 Mixture of Experts 引入风格转换语音合成转换领域。