In most cases, bilingual TTS needs to handle three types of input scripts:
first language only, second language only, and second language embedded in the
first language. In the latter two situations, the pronunciation and intonation
of the second language are usually quite different due to the influence of the
first language. Therefore, it is a big challenge to accurately model the
pronunciation and intonation of the second language in different contexts
without mutual interference. This paper builds a Mandarin-English TTS system to
acquire more standard spoken English speech from a monolingual Chinese speaker.
We introduce phonology embedding to capture the English differences between
different phonology. Embedding mask is applied to language embedding for
distinguishing information between different languages and to phonology
embedding for focusing on English expression. We specially design an embedding
strength modulator to capture the dynamic strength of language and phonology.
Experiments show that our approach can produce significantly more natural and
standard spoken English speech of the monolingual Chinese speaker. From
analysis, we find that suitable phonology control contributes to better
performance in different scenarios.

该研究构建了一个汉英文字转语音系统，引入了语音学嵌入以捕捉不同语音之间的区别，并在嵌入强度的调节和合适的语音控制方面进行设计，以便在不同情境下产生更为自然标准的英语语音。

使用动态语言与音韵嵌入改善双语 TTS

Improve Bilingual TTS Using Dynamic Language and Phonology Embedding

Training a code-switching end-to-end automatic speech recognition (ASR) model
normally requires a large amount of data, while code-switching data is often
limited. In this paper, three novel approaches are proposed for code-switching
data augmentation. Specifically, they are audio splicing with the existing
code-switching data, and TTS with new code-switching texts generated by word
translation or word insertion. Our experiments on 200 hours Mandarin-English
code-switching dataset show that all the three proposed approaches yield
significant improvements on code-switching ASR individually. Moreover, all the
proposed approaches can be combined with recent popular SpecAugment, and an
addition gain can be obtained. WER is significantly reduced by relative 24.0%
compared to the system without any data augmentation, and still relative 13.0%
gain compared to the system with only SpecAugment

本文提出了三种针对语种转换数据的增强方法，即音频接合、TTS 和单词转换或插入生成语料，并展示了这些方法对基于端到端自动语音识别系统（ASR）的语种转换系统的性能提升，这些方法可结合近期热门技术 SpecAugment 使用，相对于没有数据增强的系统，WER 显著降低了 24.0％，相对于只有 SpecAugment 的系统还有 13.0％的收益。

端到端混合语音识别的数据增强

Data Augmentation for End-to-end Code-switching Speech Recognition

Code-switching (CS) refers to a linguistic phenomenon where a speaker uses
different languages in an utterance or between alternating utterances. In this
work, we study end-to-end (E2E) approaches to the Mandarin-English
code-switching speech recognition (CSSR) task. We first examine the
effectiveness of using data augmentation and byte-pair encoding (BPE) subword
units. More importantly, we propose a multitask learning recipe, where a
language identification task is explicitly learned in addition to the E2E
speech recognition task. Furthermore, we introduce an efficient word vocabulary
expansion method for language modeling to alleviate data sparsity issues under
the code-switching scenario. Experimental results on the SEAME data, a
Mandarin-English CS corpus, demonstrate the effectiveness of the proposed
methods.

该论文以声音识别为例，探讨行话中的语言混杂现象，并提出一种多任务学习的解决方案，并针对代码切换情境下的数据稀疏问题提出了词汇扩展方法，并在 SEAME 数据上进行了实验验证。

汉英混合口语端到端语音识别的解决方案

On the End-to-End Solution to Mandarin-English Code-switching Speech  Recognition

Code-switching speech recognition has attracted an increasing interest
recently, but the need for expert linguistic knowledge has always been a big
issue. End-to-end automatic speech recognition (ASR) simplifies the building of
ASR systems considerably by predicting graphemes or characters directly from
acoustic input. In the mean time, the need of expert linguistic knowledge is
also eliminated, which makes it an attractive choice for code-switching ASR.
This paper presents a hybrid CTC-Attention based end-to-end Mandarin-English
code-switching (CS) speech recognition system and studies the effect of hybrid
CTC-Attention based models, different modeling units, the inclusion of language
identification and different decoding strategies on the task of code-switching
ASR. On the SEAME corpus, our system achieves a mixed error rate (MER) of
34.24%.

本文通过混合 CTC-Attention 模型，不同建模单元，语言识别以及不同解码策略等对汉英混合口语识别任务进行了研究，并在 SEAME 语音库上实现了 34.24% 的混淆误差率。