Recently, speech representation learning has improved many speech-related
tasks such as speech recognition, speech classification, and speech-to-text
translation. However, all the above tasks are in the direction of speech
understanding, but for the inverse direction, speech synthesis, the potential
of representation learning is yet to be realized, due to the challenging nature
of generating high-quality speech. To address this problem, we propose our
framework, Alignment-Aware Acoustic-Text Pretraining (A$^3$T), which
reconstructs masked acoustic signals with text input and acoustic-text
alignment during training. In this way, the pretrained model can generate high
quality reconstructed spectrogram, which can be applied to the speech editing
and unseen speaker TTS directly. Experiments show A$^3$T outperforms SOTA
models on speech editing, and improves multi-speaker speech synthesis without
the external speaker verification model.

该研究提出了一种名为 A³T 的框架，通过将文本输入与声学 - 文本对齐结合，训练出预训练模型来生成高质量的重构语谱图，以实现有声编辑和无外部说话人验证模型的多说话人语音合成。