Contemporary neural speech synthesis models have indeed demonstrated
remarkable proficiency in synthetic speech generation as they have attained a
level of quality comparable to that of human-produced speech. Nevertheless, it
is important to note that these achievements have predominantly been verified
within the context of high-resource languages such as English. Furthermore, the
Tacotron and FastSpeech variants show substantial pausing errors when applied
to the Korean language, which affects speech perception and naturalness. In
order to address the aforementioned issues, we propose a novel framework that
incorporates comprehensive modeling of both syntactic and acoustic cues that
are associated with pausing patterns. Remarkably, our framework possesses the
capability to consistently generate natural speech even for considerably more
extended and intricate out-of-domain (OOD) sentences, despite its training on
short audio clips. Architectural design choices are validated through
comparisons with baseline models and ablation studies using subjective and
objective metrics, thus confirming model performance.

通过综合建模句法和声学线索，提出了一种能够生成自然语音的新框架，有效解决高资源语言和韩语中的停顿错误问题。

优化韩国语 TTS 停顿生成的句法和声学线索相互作用

Leveraging the Interplay Between Syntactic and Acoustic Cues for  Optimizing Korean TTS Pause Formation

The goal of expressive Text-to-speech (TTS) is to synthesize natural speech
with desired content, prosody, emotion, or timbre, in high expressiveness. Most
of previous studies attempt to generate speech from given labels of styles and
emotions, which over-simplifies the problem by classifying styles and emotions
into a fixed number of pre-defined categories. In this paper, we introduce a
new task setting, Contextual TTS (CTTS). The main idea of CTTS is that how a
person speaks depends on the particular context she is in, where the context
can typically be represented as text. Thus, in the CTTS task, we propose to
utilize such context to guide the speech synthesis process instead of relying
on explicit labels of styles and emotions. To achieve this task, we construct a
synthetic dataset and develop an effective framework. Experiments show that our
framework can generate high-quality expressive speech based on the given
context both in synthetic datasets and real-world scenarios.

本研究提出了一种新的任务设置，即什么样的语音是由特定环境所决定的，而非仅仅局限于情绪的预定义类别，为此，我们构建了一个综合数据集并开发了一个有效的框架来生成高质量的自然语音。

情境表达式语音合成

Contextual Expressive Text-to-Speech

Decoding language from brain activity is a long-awaited goal in both
healthcare and neuroscience. Major milestones have recently been reached thanks
to intracranial devices: subject-specific pipelines trained on invasive brain
responses to basic language tasks now start to efficiently decode interpretable
features (e.g. letters, words, spectrograms). However, scaling this approach to
natural speech and non-invasive brain recordings remains a major challenge.
Here, we propose a single end-to-end architecture trained with contrastive
learning across a large cohort of individuals to predict self-supervised
representations of natural speech. We evaluate our model on four public
datasets, encompassing 169 volunteers recorded with magneto- or
electro-encephalography (M/EEG), while they listened to natural speech. The
results show that our model can identify, from 3s of MEG signals, the
corresponding speech segment with up to 72.5% top-10 accuracy out of 1,594
distinct segments (and 44% top-1 accuracy), and up to 19.1% out of 2,604
segments for EEG recordings -- hence allowing the decoding of phrases absent
from the training set. Model comparison and ablation analyses show that these
performances directly benefit from our original design choices, namely the use
of (i) a contrastive objective, (ii) pretrained representations of speech and
(iii) a common convolutional architecture simultaneously trained across several
participants. Together, these results delineate a promising path to decode
natural language processing in real time from non-invasive recordings of brain
activity.

通过对大量受试者进行对比学习，使用自我监督方法设计的神经网络可以从无创脑电图数据中有效识别自由自然语言，为实现从脑电图中实时解码自然语音处理提供了有前途的途径。