This paper proposes a unified model to conduct emotion transfer, control and
prediction for sequence-to-sequence based fine-grained emotional speech
synthesis. Conventional emotional speech synthesis often needs manual labels or
reference audio to determine the emotional expressions of synthesized speech.
Such coarse labels cannot control the details of speech emotion, often
resulting in an averaged emotion expression delivery, and it is also hard to
choose suitable reference audio during inference. To conduct fine-grained
emotion expression generation, we introduce phoneme-level emotion strength
representations through a learned ranking function to describe the local
emotion details, and the sentence-level emotion category is adopted to render
the global emotions of synthesized speech. With the global render and local
descriptors of emotions, we can obtain fine-grained emotion expressions from
reference audio via its emotion descriptors (for transfer) or directly from
phoneme-level manual labels (for control). As for the emotional speech
synthesis with arbitrary text inputs, the proposed model can also predict
phoneme-level emotion expressions from texts, which does not require any
reference audio or manual label.

本文提出了一种用于序列到序列的细粒度情感语音合成的统一模型，通过学习到的排名函数引入音素级情感强度表示来描述本地情感细节，并采用句子级情感类别来呈现合成语音的全局情感。