Self-supervised learning (SSL) leverages large datasets of unlabeled speech
to reach impressive performance with reduced amounts of annotated data. The
high number of proposed approaches fostered the emergence of comprehensive
benchmarks that evaluate their performance on a set of downstream tasks
exploring various aspects of the speech signal. However, while the number of
considered tasks has been growing, most proposals rely upon a single downstream
architecture that maps the frozen SSL representations to the task labels. This
study examines how benchmarking results are affected by changes in the probing
head architecture. Interestingly, we found that altering the downstream
architecture structure leads to significant fluctuations in the performance
ranking of the evaluated models. Against common practices in speech SSL
benchmarking, we evaluate larger-capacity probing heads, showing their impact
on performance, inference costs, generalization and multi-level feature
exploitation.

自监督学习利用大规模的无标签语音数据集，在少量标注数据的情况下达到令人印象深刻的性能。本研究探讨了在探测头结构改变时，基准测试结果如何受到影响。通过评估大容量的探测头，显示了它们对性能、推理成本、泛化能力和多级特征利用的影响。