Speech-to-Text Translation (S2TT) has typically been addressed with cascade
systems, where speech recognition systems generate a transcription that is
subsequently passed to a translation model. While there has been a growing
interest in developing direct speech translation systems to avoid propagating
errors and losing non-verbal content, prior work in direct S2TT has struggled
to conclusively establish the advantages of integrating the acoustic signal
directly into the translation process. This work proposes using contrastive
evaluation to quantitatively measure the ability of direct S2TT systems to
disambiguate utterances where prosody plays a crucial role. Specifically, we
evaluated Korean-English translation systems on a test set containing
wh-phrases, for which prosodic features are necessary to produce translations
with the correct intent, whether it's a statement, a yes/no question, a
wh-question, and more. Our results clearly demonstrate the value of direct
translation systems over cascade translation models, with a notable 12.9%
improvement in overall accuracy in ambiguous cases, along with up to a 15.6%
increase in F1 scores for one of the major intent categories. To the best of
our knowledge, this work stands as the first to provide quantitative evidence
that direct S2TT models can effectively leverage prosody. The code for our
evaluation is openly accessible and freely available for review and
utilisation.

直接语音到文本翻译系统可以有效利用韵律，具有比级联翻译模型更高的准确性和意图分类效果。

级联和直接语音到文本翻译中的韵律学：以韩语 Wh 短语为例的案例研究

Prosody in Cascade and Direct Speech-to-Text Translation: a case study  on Korean Wh-Phrases

This paper introduces StutterNet, a novel deep learning based stuttering
detection capable of detecting and identifying various types of disfluencies.
Most of the existing work in this domain uses automatic speech recognition
(ASR) combined with language models for stuttering detection. Compared to the
existing work, which depends on the ASR module, our method relies solely on the
acoustic signal. We use a time-delay neural network (TDNN) suitable for
capturing contextual aspects of the disfluent utterances. We evaluate our
system on the UCLASS stuttering dataset consisting of more than 100 speakers.
Our method achieves promising results and outperforms the state-of-the-art
residual neural network based method. The number of trainable parameters of the
proposed method is also substantially less due to the parameter sharing scheme
of TDNN.

本文介绍了 StutterNet，一种新颖的基于深度学习的口吃检测方法，仅依赖于声学信号，使用适用于捕获失聚言语语境方面的时延神经网络，并在包含 100 多个说话者的 UCLASS 口吃数据集上验证，结果表明该方法优于现有方法，参数少并且性能良好。