Text to speech (TTS) has been broadly used to synthesize natural and
intelligible speech in different scenarios. Deploying TTS in various end
devices such as mobile phones or embedded devices requires extremely small
memory usage and inference latency. While non-autoregressive TTS models such as
FastSpeech have achieved significantly faster inference speed than
autoregressive models, their model size and inference latency are still large
for the deployment in resource constrained devices. In this paper, we propose
LightSpeech, which leverages neural architecture search~(NAS) to automatically
design more lightweight and efficient models based on FastSpeech. We first
profile the components of current FastSpeech model and carefully design a novel
search space containing various lightweight and potentially effective
architectures. Then NAS is utilized to automatically discover well performing
architectures within the search space. Experiments show that the model
discovered by our method achieves 15x model compression ratio and 6.5x
inference speedup on CPU with on par voice quality. Audio demos are provided at
this https URL

本文提出了一种名为 LightSpeech 的 TTS 模型，该模型通过利用神经架构搜索自动设计出轻量级的、高效的模型，并在不损失语音质量的情况下，将模型压缩率提高了 15 倍、推断速度提高了 6.5 倍。

LightSpeech: 基于神经架构搜索的轻量级快速语音合成

LightSpeech: Lightweight and Fast Text to Speech with Neural  Architecture Search

This paper presents XiaoiceSing, a high-quality singing voice synthesis
system which employs an integrated network for spectrum, F0 and duration
modeling. We follow the main architecture of FastSpeech while proposing some
singing-specific design: 1) Besides phoneme ID and position encoding, features
from musical score (e.g.note pitch and length) are also added. 2) To attenuate
off-key issues, we add a residual connection in F0 prediction. 3) In addition
to the duration loss of each phoneme, the duration of all the phonemes in a
musical note is accumulated to calculate the syllable duration loss for rhythm
enhancement. Experiment results show that XiaoiceSing outperforms the baseline
system of convolutional neural networks by 1.44 MOS on sound quality, 1.18 on
pronunciation accuracy and 1.38 on naturalness respectively. In two A/B tests,
the proposed F0 and duration modeling methods achieve 97.3% and 84.3%
preference rate over baseline respectively, which demonstrates the overwhelming
advantages of XiaoiceSing.

该论文介绍了 XiaoiceSing，这是一个高质量的歌唱合成系统，它采用集成网络进行频谱、 F0 和持续时间建模。该系统在保留 FastSpeech 的主要框架的同时提出了一些特定于歌唱的设计，通过多种实验验证该系统表现优异。

XiaoiceSing: 高质量集成歌声合成系统

XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis  System

Non-autoregressive text to speech (TTS) models such as FastSpeech can
synthesize speech significantly faster than previous autoregressive models with
comparable quality. The training of FastSpeech model relies on an
autoregressive teacher model for duration prediction (to provide more
information as input) and knowledge distillation (to simplify the data
distribution in output), which can ease the one-to-many mapping problem (i.e.,
multiple speech variations correspond to the same text) in TTS. However,
FastSpeech has several disadvantages: 1) the teacher-student distillation
pipeline is complicated and time-consuming, 2) the duration extracted from the
teacher model is not accurate enough, and the target mel-spectrograms distilled
from teacher model suffer from information loss due to data simplification,
both of which limit the voice quality. In this paper, we propose FastSpeech 2,
which addresses the issues in FastSpeech and better solves the one-to-many
mapping problem in TTS by 1) directly training the model with ground-truth
target instead of the simplified output from teacher, and 2) introducing more
variation information of speech (e.g., pitch, energy and more accurate
duration) as conditional inputs. Specifically, we extract duration, pitch and
energy from speech waveform and directly take them as conditional inputs in
training and use predicted values in inference. We further design FastSpeech
2s, which is the first attempt to directly generate speech waveform from text
in parallel, enjoying the benefit of fully end-to-end inference. Experimental
results show that 1) FastSpeech 2 achieves a 3x training speed-up over
FastSpeech, and FastSpeech 2s enjoys even faster inference speed; 2) FastSpeech
2 and 2s outperform FastSpeech in voice quality, and FastSpeech 2 can even
surpass autoregressive models. Audio samples are available at
this https URL

提出了 FastSpeech 2，直接使用语音波形从文本中生成语音，并且通过使用更多的变化信息作为条件输入，解决了非自回归文本到语音模型中的一对多映射问题，从而实现更高的语音质量。

FastSpeech 2: 快速高質量的端到端文本轉語音

FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

Neural network based end-to-end text to speech (TTS) has significantly
improved the quality of synthesized speech. Prominent methods (e.g., Tacotron
2) usually first generate mel-spectrogram from text, and then synthesize speech
from the mel-spectrogram using vocoder such as WaveNet. Compared with
traditional concatenative and statistical parametric approaches, neural network
based end-to-end models suffer from slow inference speed, and the synthesized
speech is usually not robust (i.e., some words are skipped or repeated) and
lack of controllability (voice speed or prosody control). In this work, we
propose a novel feed-forward network based on Transformer to generate
mel-spectrogram in parallel for TTS. Specifically, we extract attention
alignments from an encoder-decoder based teacher model for phoneme duration
prediction, which is used by a length regulator to expand the source phoneme
sequence to match the length of the target mel-spectrogram sequence for
parallel mel-spectrogram generation. Experiments on the LJSpeech dataset show
that our parallel model matches autoregressive models in terms of speech
quality, nearly eliminates the problem of word skipping and repeating in
particularly hard cases, and can adjust voice speed smoothly. Most importantly,
compared with autoregressive Transformer TTS, our model speeds up
mel-spectrogram generation by 270x and the end-to-end speech synthesis by 38x.
Therefore, we call our model FastSpeech.

本研究提出了一种基于 Transformer 模型、并行生成梅尔频谱的快速语音合成模型 FastSpeech，它在语音质量、稳定性、可控性和速度方面均具备相应优点。