In this work, we take on the challenging task of building a single
text-to-speech synthesis system that is capable of generating speech in over
7000 languages, many of which lack sufficient data for traditional TTS
development. By leveraging a novel integration of massively multilingual
pretraining and meta learning to approximate language representations, our
approach enables zero-shot speech synthesis in languages without any available
data. We validate our system's performance through objective measures and human
evaluation across a diverse linguistic landscape. By releasing our code and
models publicly, we aim to empower communities with limited linguistic
resources and foster further innovation in the field of speech technology.

通过大规模多语言预训练和元学习，我们构建了一个能够在超过 7000 种语言中生成语音的单一文本转语音合成系统，并通过客观评估和人工评价验证了该系统在多样化语言环境下的性能。通过公开发布我们的代码和模型，我们旨在为语言资源有限的社区提供支持，并在语音技术领域促进进一步的创新。

元学习文本转语音在 7000 多种语言中的应用

Meta Learning Text-to-Speech Synthesis in over 7000 Languages

Recent progress in large-scale zero-shot speech synthesis has been
significantly advanced by language models and diffusion models. However, the
generation process of both methods is slow and computationally intensive.
Efficient speech synthesis using a lower computing budget to achieve quality on
par with previous work remains a significant challenge. In this paper, we
present FlashSpeech, a large-scale zero-shot speech synthesis system with
approximately 5\% of the inference time compared with previous work.
FlashSpeech is built on the latent consistency model and applies a novel
adversarial consistency training approach that can train from scratch without
the need for a pre-trained diffusion model as the teacher. Furthermore, a new
prosody generator module enhances the diversity of prosody, making the rhythm
of the speech sound more natural. The generation processes of FlashSpeech can
be achieved efficiently with one or two sampling steps while maintaining high
audio quality and high similarity to the audio prompt for zero-shot speech
generation. Our experimental results demonstrate the superior performance of
FlashSpeech. Notably, FlashSpeech can be about 20 times faster than other
zero-shot speech synthesis systems while maintaining comparable performance in
terms of voice quality and similarity. Furthermore, FlashSpeech demonstrates
its versatility by efficiently performing tasks like voice conversion, speech
editing, and diverse speech sampling. Audio samples can be found in
this https URL

闪电语音是一种大规模零样本语音合成系统，使用比之前的方法推理时间约占 5％，具有高音质和高相似性的性能，同时实现了语音处理的多样性。

FlashSpeech：高效零射击语音合成

FlashSpeech: Efficient Zero-Shot Speech Synthesis

Large language models (LLM)-based speech synthesis has been widely adopted in
zero-shot speech synthesis. However, they require a large-scale data and
possess the same limitations as previous autoregressive speech models,
including slow inference speed and lack of robustness. This paper proposes
HierSpeech++, a fast and strong zero-shot speech synthesizer for text-to-speech
(TTS) and voice conversion (VC). We verified that hierarchical speech synthesis
frameworks could significantly improve the robustness and expressiveness of the
synthetic speech. Furthermore, we significantly improve the naturalness and
speaker similarity of synthetic speech even in zero-shot speech synthesis
scenarios. For text-to-speech, we adopt the text-to-vec framework, which
generates a self-supervised speech representation and an F0 representation
based on text representations and prosody prompts. Then, HierSpeech++ generates
speech from the generated vector, F0, and voice prompt. We further introduce a
high-efficient speech super-resolution framework from 16 kHz to 48 kHz. The
experimental results demonstrated that the hierarchical variational autoencoder
could be a strong zero-shot speech synthesizer given that it outperforms
LLM-based and diffusion-based models. Moreover, we achieved the first
human-level quality zero-shot speech synthesis. Audio samples and source code
are available at this https URL

HierSpeech++ 是一种快速而强大的零样本语音合成器，适用于文本转语音（TTS）和语音转换（VC），通过采用层次化语音合成框架，显著提高了合成语音的健壮性、表达力和自然度，在零样本语音合成场景中实现了人类级质量。

HierSpeech++：通过分层变分推断在语音的语义和声学表示之间建立联系以进行零样本语音合成

HierSpeech++: Bridging the Gap between Semantic and Acoustic  Representation of Speech by Hierarchical Variational Inference for Zero-shot  Speech Synthesis