Toxicity classification for voice heavily relies on the semantic content of
speech. We propose a novel framework that utilizes cross-modal learning to
integrate the semantic embedding of text into a multilabel speech toxicity
classifier during training. This enables us to incorporate textual information
during training while still requiring only audio during inference. We evaluate
this classifier on large-scale datasets with real-world characteristics to
validate the effectiveness of this framework. Through ablation studies, we
demonstrate that general-purpose semantic text embeddings are rich and aligned
with speech for toxicity classification purposes. Conducting experiments across
multiple languages at scale, we show improvements in voice toxicity
classification across five languages and different toxicity categories.

利用跨模态学习将文本的语义嵌入集成到多标签语音毒性分类器中，该分类器仅在训练期间需要文本信息，推理时仅需要音频信息，并通过大规模数据集在真实环境下验证了此框架的有效性，同时通过消融研究证明了通用语义文本嵌入对于毒性分类目的的丰富性和与语音的一致性，通过规模化跨多种语言进行实验，展示了在五种语言和不同毒性类别中语音毒性分类的改进。

通过语音文本对齐提升多语言语音有害性检测

Enhancing Multilingual Voice Toxicity Detection with Speech-Text  Alignment

We present a diachronic acoustic analysis of the voice of 1023 speakers from
French media archives. The speakers are spread across 32 categories based on
four periods (years 1955/56, 1975/76, 1995/96, 2015/16), four age groups
(20-35; 36-50; 51-65, >65), and two genders. The fundamental frequency ($F_0$)
and the first four formants (F1-4) were estimated. Procedures used to ensure
the quality of these estimations on heterogeneous data are described. From each
speaker's $F_0$ distribution, the base-$F_0$ value was calculated to estimate
the register. Average vocal tract length was estimated from formant
frequencies. Base-$F_0$ and vocal tract length were fit by linear mixed models
to evaluate how they may have changed across time periods and genders,
corrected for age effects. Results show an effect of the period with a tendency
to lower voices, independently of gender. A lowering of pitch is observed with
age for female but not male speakers.

通过对 1023 名来自法国媒体档案的讲话者进行历时声学分析，我们发现在四个不同时期、四个年龄组群和两性别范围内，声音频率和共振峰频率会发生变化。研究结果表明，尽管受性别影响较小，但不同时期的说话声音具有偏低的倾向，女性说话者在年龄增长时出现音调降低的现象。

法国音频视听媒体中性别和年龄声音的历时演变

Evolution of Voices in French Audiovisual Media Across Genders and Age  in a Diachronic Perspective

There has been a range of studies of how autism is displayed in voice,
speech, and language. We analyse studies from the biomedical, as well as the
psychological domain, but also from the NLP domain in order to find linguistic,
prosodic and acoustic cues that could indicate autism. Our survey looks at all
three domains. We define autism and which comorbidities might influence the
correct detection of the disorder. We especially look at observations such as
verbal and semantic fluency, prosodic features, but also disfluencies and
speaking rate. We also show word-based approaches and describe machine learning
and transformer-based approaches both on the audio data as well as the
transcripts. Lastly, we conclude, while there already is a lot of research,
female patients seem to be severely under-researched. Also, most NLP research
focuses on traditional machine learning methods instead of transformers which
could be beneficial in this context. Additionally, we were unable to find
research combining both features from audio and transcripts.

通过分析来自生物医学、心理学和自然语言处理领域的研究，我们寻找能够指示自闭症的语言、韵律和声学线索。此调查涵盖了自闭症的定义、可能影响正确诊断的共病疾病，以及诸如语言流畅性、韵律特征、不流畅性和说话速度等观察结果。我们还介绍了基于词汇的方法，并描述了对音频数据和文本的机器学习和基于转换器的方法。最后，我们得出结论，目前已有大量研究，但女性患者的研究仍然非常有限，而且大多数自然语言处理研究侧重于传统的机器学习方法，而非在这一领域有潜力的转换器。另外，我们未能找到将音频和文本特征结合起来的研究。