Self-supervised learning (SSL) for speech representation has been
successfully applied in various downstream tasks, such as speech and speaker
recognition. More recently, speech SSL models have also been shown to be
beneficial in advancing spoken language understanding tasks, implying that the
SSL models have the potential to learn not only acoustic but also linguistic
information. In this paper, we aim to clarify if speech SSL techniques can well
capture linguistic knowledge. For this purpose, we introduce SpeechGLUE, a
speech version of the General Language Understanding Evaluation (GLUE)
benchmark. Since GLUE comprises a variety of natural language understanding
tasks, SpeechGLUE can elucidate the degree of linguistic ability of speech SSL
models. Experiments demonstrate that speech SSL models, although inferior to
text-based SSL models, perform better than baselines, suggesting that they can
acquire a certain amount of general linguistic knowledge from just unlabeled
speech data.

本研究探讨了自监督学习技术对于语音表示及其所携带的语言信息的捕捉能力，并通过 SpeechGLUE 基准测试说明了此技术在语言理解任务方面表现不如基于文本的自监督学习，但优于基准模型，展示了它从无标签的语音数据中能够获取特定数量的语言能力。