Speech-to-Speech and Speech-to-Text translation are currently dynamic areas
of research. To contribute to these fields, we present SpeechAlign, a framework
to evaluate the underexplored field of source-target alignment in speech
models. Our framework has two core components. First, to tackle the absence of
suitable evaluation datasets, we introduce the Speech Gold Alignment dataset,
built upon a English-German text translation gold alignment dataset. Secondly,
we introduce two novel metrics, Speech Alignment Error Rate (SAER) and
Time-weighted Speech Alignment Error Rate (TW-SAER), to evaluate alignment
quality in speech models. By publishing SpeechAlign we provide an accessible
evaluation framework for model assessment, and we employ it to benchmark
open-source Speech Translation models.

SpeechAlign 是一个框架，用于评估语音模型中源 - 目标对齐领域的尚未充分探索的领域，并提供了两个新的度量标准来评估语音模型的对齐质量，通过发布 SpeechAlign 来为模型评估提供一个可访问的评估框架，并用它来评估开源的语音翻译模型。