Cross-speaker style transfer in speech synthesis aims at transferring a style
from source speaker to synthesized speech of a target speaker's timbre. In most
previous methods, the synthesized fine-grained prosody features often represent
the source speaker's average style, similar to the one-to-many problem(i.e.,
multiple prosody variations correspond to the same text). In response to this
problem, a strength-controlled semi-supervised style extractor is proposed to
disentangle the style from content and timbre, improving the representation and
interpretability of the global style embedding, which can alleviate the
one-to-many mapping and data imbalance problems in prosody prediction. A
hierarchical prosody predictor is proposed to improve prosody modeling. We find
that better style transfer can be achieved by using the source speaker's
prosody features that are easily predicted. Additionally, a
speaker-transfer-wise cycle consistency loss is proposed to assist the model in
learning unseen style-timbre combinations during the training phase.
Experimental results show that the method outperforms the baseline. We provide
a website with audio samples.

提出了一种强度可控半监督风格提取器，用于解开语音中的风格、内容和音色，同时设计了分层韵律预测器以提高其音韵建模，并提出跨发话人循环一致性损失作为训练阶段辅助模型学习未见过的风格 - 音色组合，并展示它的效果优于基线方法。

使用半监督风格提取器和分层建模提高跨说话人风格转移的韵律表现在语音合成中

Improving Prosody for Cross-Speaker Style Transfer by Semi-Supervised Style Extractor and Hierarchical Modeling in Speech Synthesis

Cross-speaker style transfer aims to extract the speech style of the given
reference speech, which can be reproduced in the timbre of arbitrary target
speakers. Existing methods on this topic have explored utilizing
utterance-level style labels to perform style transfer via either global or
local scale style representations. However, audiobook datasets are typically
characterized by both the local prosody and global genre, and are rarely
accompanied by utterance-level style labels. Thus, properly transferring the
reading style across different speakers remains a challenging task. This paper
aims to introduce a chunk-wise multi-scale cross-speaker style model to capture
both the global genre and the local prosody in audiobook speeches. Moreover, by
disentangling speaker timbre and style with the proposed switchable adversarial
classifiers, the extracted reading style is made adaptable to the timbre of
different speakers. Experiment results confirm that the model manages to
transfer a given reading style to new target speakers. With the support of
local prosody and global genre type predictor, the potentiality of the proposed
method in multi-speaker audiobook generation is further revealed.

通过提出的分块多尺度跨说话人风格模型以及可切换式对抗分类器，本文旨在解决跨说话人的阅读风格转移问题，并证实其在多人语音书籍生成中的潜力。

基于有声书数据的跨说话人阅读风格转移

Towards Cross-speaker Reading Style Transfer on Audiobook Dataset

Expressive text-to-speech has shown improved performance in recent years.
However, the style control of synthetic speech is often restricted to discrete
emotion categories and requires training data recorded by the target speaker in
the target style. In many practical situations, users may not have reference
speech recorded in target emotion but still be interested in controlling speech
style just by typing text description of desired emotional style. In this
paper, we propose a text-based interface for emotional style control and
cross-speaker style transfer in multi-speaker TTS. We propose the bi-modal
style encoder which models the semantic relationship between text description
embedding and speech style embedding with a pretrained language model. To
further improve cross-speaker style transfer on disjoint, multi-style datasets,
we propose the novel style loss. The experimental results show that our model
can generate high-quality expressive speech even in unseen style.

该论文提出了一种基于文本的情感风格控制接口和跨说话人风格转移方法，利用双模态风格编码器以及新的风格损失函数，实现高质量的表达性语音合成。