We present ProsAudit, a benchmark in English to assess structural prosodic
knowledge in self-supervised learning (SSL) speech models. It consists of two
subtasks, their corresponding metrics, an evaluation dataset. In the
protosyntax task, the model must correctly identify strong versus weak prosodic
boundaries. In the lexical task, the model needs to correctly distinguish
between pauses inserted between words and within words. We also provide human
evaluation scores on this benchmark. We evaluated a series of SSL models and
found that they were all able to perform above chance on both tasks, even when
trained on an unseen language. However, non-native models performed
significantly worse than native ones on the lexical task, highlighting the
importance of lexical knowledge in this task. We also found a clear effect of
size with models trained on more data performing better in the two subtasks.

本文介绍了一个叫 ProsAudit 的英文基准测试，旨在评估自我监督学习（SSL）语音模型中的结构韵律知识，它由两个子任务、相应的指标和评估数据集组成。通过对一系列 SSL 模型的评估，发现这些模型在两个子任务上表现良好，特别地，大型模型在这两个子任务中表现更好。同时本文也强调了词汇知识在这一任务中的重要性。