We introduce Parallel Paraphrasing ($\text{Para}_\text{both}$), an
augmentation method for translation metrics making use of automatic
paraphrasing of both the reference and hypothesis. This method counteracts the
typically misleading results of speech translation metrics such as WER, CER,
and BLEU if only a single reference is available. We introduce two new datasets
explicitly created to measure the quality of metrics intended to be applied to
Swiss German speech-to-text systems. Based on these datasets, we show that we
are able to significantly improve the correlation with human quality perception
if our method is applied to commonly used metrics.

本文介绍了一种叫做 Parallel Paraphrasing 的翻译度量扩展方法，它利用参考和猜想的自动改写。这种方法对于只有一个参考的语音翻译度量（如 WER，CER 和 BLEU）的误导性结果进行了抵消。我们引入了两个新数据集，专门用于评估适用于瑞士德语语音转文本系统的测试的质量。基于这些数据集，我们展示了如果我们将该方法应用于常用测试中，我们能够显著提高与人类质量感知的相关性。

提升语音翻译指标

Improving Metrics for Speech Translation

We present an in-depth evaluation of four commercially available
Speech-to-Text (STT) systems for Swiss German. The systems are anonymized and
referred to as system a-d in this report. We compare the four systems to our
STT model, referred to as FHNW from hereon after, and provide details on how we
trained our model. To evaluate the models, we use two STT datasets from
different domains. The Swiss Parliament Corpus (SPC) test set and a private
dataset in the news domain with an even distribution across seven dialect
regions. We provide a detailed error analysis to detect the three systems'
strengths and weaknesses. This analysis is limited by the characteristics of
the two test sets. Our model scored the highest bilingual evaluation understudy
(BLEU) on both datasets. On the SPC test set, we obtain a BLEU score of 0.607,
whereas the best commercial system reaches a BLEU score of 0.509. On our
private test set, we obtain a BLEU score of 0.722 and the best commercial
system a BLEU score of 0.568.

针对四个商业可用的瑞士德语语音转文本系统和 FHNW 模型展开深入评估，并使用瑞士议会语料库和新闻域中的私人数据集来评估这些模型，并提供了有关模型培训的详细信息，评估了这些模型的强​​弱点并对其进行了详细的错误分析。