Recent language model-based text-to-speech (TTS) frameworks demonstrate
scalability and in-context learning capabilities. However, they suffer from
robustness issues due to the accumulation of errors in speech unit predictions
during autoregressive language modeling. In this paper, we propose a phonetic
enhanced language modeling method to improve the performance of TTS models. We
leverage self-supervised representations that are phonetically rich as the
training target for the autoregressive language model. Subsequently, a
non-autoregressive model is employed to predict discrete acoustic codecs that
contain fine-grained acoustic details. The TTS model focuses solely on
linguistic modeling during autoregressive training, thereby reducing the error
propagation that occurs in non-autoregressive training. Both objective and
subjective evaluations validate the effectiveness of our proposed method.

我们提出了一种声学增强的语言建模方法，通过使用语音单元预测中的错误积累来改善 TTS 模型的性能，通过自我监督表示来作为自回归语言模型的训练目标，并采用非自回归模型来预测包含细粒度声学细节的离散声学编解码器，以减少非自回归训练中的错误传播，客观和主观评估验证了我们提出方法的有效性。