Self-supervised models for speech processing form representational spaces
without using any external labels. Increasingly, they appear to be a feasible
way of at least partially eliminating costly manual annotations, a problem of
particular concern for low-resource languages. But what kind of
representational spaces do these models construct? Human perception specializes
to the sounds of listeners' native languages. Does the same thing happen in
self-supervised models? We examine the representational spaces of three kinds
of state-of-the-art self-supervised models: wav2vec 2.0, HuBERT and contrastive
predictive coding (CPC), and compare them with the perceptual spaces of
French-speaking and English-speaking human listeners, both globally and taking
account of the behavioural differences between the two language groups. We show
that the CPC model shows a small native language effect, but that wav2vec 2.0
and HuBERT seem to develop a universal speech perception space which is not
language specific. A comparison against the predictions of supervised phone
recognisers suggests that all three self-supervised models capture relatively
fine-grained perceptual phenomena, while supervised models are better at
capturing coarser, phone-level, effects of listeners' native language, on
perception.

通过研究 wav2vec 2.0，HuBERT 和 CPC 三种最先进的自监督模型的表示空间，与法语和英语人类听众的感知空间作比较，我们发现 CPC 模型具有微小的母语语言效应，但 wav2vec 2.0 和 HuBERT 似乎发展出了一种通用的语音感知空间，而不是针对特定语言的。与受监督的电话识别器的预测比较表明，所有三种自监督的模型捕获了相对精细的知觉现象，而受监督的模型则更擅长捕获听众母语的更粗的电话级效应。