State of the art (SOTA) neural text to speech (TTS) models can generate
natural-sounding synthetic voices. These models are characterized by large
memory footprints and substantial number of operations due to the long-standing
focus on speech quality with cloud inference in mind. Neural TTS models are
generally not designed to perform standalone speech syntheses on
resource-constrained and no Internet access edge devices. In this work, an
efficient neural TTS called EfficientSpeech that synthesizes speech on an ARM
CPU in real-time is proposed. EfficientSpeech uses a shallow non-autoregressive
pyramid-structure transformer forming a U-Network. EfficientSpeech has 266k
parameters and consumes 90 MFLOPS only or about 1% of the size and amount of
computation in modern compact models such as Mixer-TTS. EfficientSpeech
achieves an average mel generation real-time factor of 104.3 on an RPi4. Human
evaluation shows only a slight degradation in audio quality as compared to
FastSpeech2.

本文提出了一种名为 EfficientSpeech 的神经文本转语音模型，它可以在资源受限且没有互联网访问的边缘设备上使用 ARM CPU 实时合成语音，并且相比现代紧凑型模型，参数和计算量仅为 1％，可实现平均 104.3 倍的实时因素，人类评估显示与 FastSpeech2 相比，音频质量仅略有下降。

EfficientSpeech: 一个本地化文本到语音模型

EfficientSpeech: An On-Device Text to Speech Model

This paper describes the Microsoft end-to-end neural text to speech (TTS)
system: DelightfulTTS for Blizzard Challenge 2021. The goal of this challenge
is to synthesize natural and high-quality speech from text, and we approach
this goal in two perspectives: The first is to directly model and generate
waveform in 48 kHz sampling rate, which brings higher perception quality than
previous systems with 16 kHz or 24 kHz sampling rate; The second is to model
the variation information in speech through a systematic design, which improves
the prosody and naturalness. Specifically, for 48 kHz modeling, we predict 16
kHz mel-spectrogram in acoustic model, and propose a vocoder called HiFiNet to
directly generate 48 kHz waveform from predicted 16 kHz mel-spectrogram, which
can better trade off training efficiency, modelling stability and voice
quality. We model variation information systematically from both explicit
(speaker ID, language ID, pitch and duration) and implicit (utterance-level and
phoneme-level prosody) perspectives: 1) For speaker and language ID, we use
lookup embedding in training and inference; 2) For pitch and duration, we
extract the values from paired text-speech data in training and use two
predictors to predict the values in inference; 3) For utterance-level and
phoneme-level prosody, we use two reference encoders to extract the values in
training, and use two separate predictors to predict the values in inference.
Additionally, we introduce an improved Conformer block to better model the
local and global dependency in acoustic model. For task SH1, DelightfulTTS
achieves 4.17 mean score in MOS test and 4.35 in SMOS test, which indicates the
effectiveness of our proposed system

本文描述了 Microsoft 的端到端神经文本语音合成系统：用于 Blizzard Challenge 2021 的 DelightfulTTS。该系统在两个方面实现了将文本合成成自然且高质量的语音：一方面是直接建模和生成 48 kHz 采样率的波形，另一方面是通过系统化设计来建模语音中的变化信息，从而提高韵律和自然度。

DelightfulTTS: 微软语音合成系统参加暴雪挑战 2021

DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard  Challenge 2021

We present BOFFIN TTS (Bayesian Optimization For FIne-tuning Neural Text To
Speech), a novel approach for few-shot speaker adaptation. Here, the task is to
fine-tune a pre-trained TTS model to mimic a new speaker using a small corpus
of target utterances. We demonstrate that there does not exist a
one-size-fits-all adaptation strategy, with convincing synthesis requiring a
corpus-specific configuration of the hyper-parameters that control fine-tuning.
By using Bayesian optimization to efficiently optimize these hyper-parameter
values for a target speaker, we are able to perform adaptation with an average
30% improvement in speaker similarity over standard techniques. Results
indicate, across multiple corpora, that BOFFIN TTS can learn to synthesize new
speakers using less than ten minutes of audio, achieving the same naturalness
as produced for the speakers used to train the base model.

使用 BOFFIN TTS，利用贝叶斯优化实现神经文本转语音的少量说话人适应，通过优化调节超级参数值，能够在小于十分钟的语音中学习新的说话人，从而实现与基础模型训练的说话人相同的自然度。