Recent self-supervised learning (SSL) models have proven to learn rich
representations of speech, which can readily be utilized by diverse downstream
tasks. To understand such utilities, various analyses have been done for speech
SSL models to reveal which and how information is encoded in the learned
representations. Although the scope of previous analyses is extensive in
acoustic, phonetic, and semantic perspectives, the physical grounding by speech
production has not yet received full attention. To bridge this gap, we conduct
a comprehensive analysis to link speech representations to articulatory
trajectories measured by electromagnetic articulography (EMA). Our analysis is
based on a linear probing approach where we measure articulatory score as an
average correlation of linear mapping to EMA. We analyze a set of SSL models
selected from the leaderboard of the SUPERB benchmark and perform further
layer-wise analyses on two most successful models, Wav2Vec 2.0 and HuBERT.
Surprisingly, representations from the recent speech SSL models are highly
correlated with EMA traces (best: r = 0.81), and only 5 minutes are sufficient
to train a linear model with high performance (r = 0.77). Our findings suggest
that SSL models learn to align closely with continuous articulations, and
provide a novel insight into speech SSL.

本文研究了自监督学习模型与电磁发音学之间的联系，发现这些模型学习到了与语音连续性运动高度相关的表示，并建议这为人工智能技术的未来发展带来了新的启示。