This paper introduces VALL-E 2, the latest advancement in neural codec
language models that marks a milestone in zero-shot text-to-speech synthesis
(TTS), achieving human parity for the first time. Based on its predecessor,
VALL-E, the new iteration introduces two significant enhancements: Repetition
Aware Sampling refines the original nucleus sampling process by accounting for
token repetition in the decoding history. It not only stabilizes the decoding
but also circumvents the infinite loop issue. Grouped Code Modeling organizes
codec codes into groups to effectively shorten the sequence length, which not
only boosts inference speed but also addresses the challenges of long sequence
modeling. Our experiments on the LibriSpeech and VCTK datasets show that VALL-E
2 surpasses previous systems in speech robustness, naturalness, and speaker
similarity. It is the first of its kind to reach human parity on these
benchmarks. Moreover, VALL-E 2 consistently synthesizes high-quality speech,
even for sentences that are traditionally challenging due to their complexity
or repetitive phrases. The advantages of this work could contribute to valuable
endeavors, such as generating speech for individuals with aphasia or people
with amyotrophic lateral sclerosis. Demos of VALL-E 2 will be posted to
this https URL

VALL-E 2 是零样本文本到语音合成领域最新的神经编解码模型，首次实现了人类水平的合成，并引入了两个重要的改进：重复感知采样和编解码编组建模。实验结果表明 VALL-E 2 在语音鲁棒性、自然程度和说话者相似性方面均超过了之前的系统，在 LibriSpeech 和 VCTK 数据集上取得了人类水平的合成效果，具有很大的潜在应用价值。