Controllable generative sequence models with the capability to extract and
replicate the style of specific examples enable many applications, including
narrating audiobooks in different voices, auto-completing and auto-correcting
written handwriting, and generating missing training samples for downstream
recognition tasks. However, under an unsupervised-style setting, typical
training algorithms for controllable sequence generative models suffer from the
training-inference mismatch, where the same sample is used as content and style
input during training but unpaired samples are given during inference. In this
paper, we tackle the training-inference mismatch encountered during
unsupervised learning of controllable generative sequence models. The proposed
method is simple yet effective, where we use a style transformation module to
transfer target style information into an unrelated style input. This method
enables training using unpaired content and style samples and thereby mitigate
the training-inference mismatch. We apply style equalization to text-to-speech
and text-to-handwriting synthesis on three datasets. We conduct thorough
evaluation, including both quantitative and qualitative user studies. Our
results show that by mitigating the training-inference mismatch with the
proposed style equalization, we achieve style replication scores comparable to
real data in our user studies.

本文介绍了解决无监督学习中可控生成序列模型的训练 - 推断不匹配问题的方法，该方法利用样式转换模块将目标样式信息转移至无关样式输入进行训练，实现了在未配对的内容和样式样本下进行训练并缓解了训练 - 推断不匹配问题，通过文本到语音合成和文本到手写合成的实验验证了提出的样式均衡方法，结果表明在用户研究中，该方法实现了与实际数据相当的风格复制分数。