Speaker identity plays a significant role in human communication and is being
increasingly used in societal applications, many through advances in machine
learning. Speaker identity perception is an essential cognitive phenomenon that
can be broadly reduced to two main tasks: recognizing a voice or discriminating
between voices. Several studies have attempted to identify acoustic correlates
of identity perception to pinpoint salient parameters for such a task. Unlike
other communicative social signals, most efforts have yielded inefficacious
conclusions. Furthermore, current neurocognitive models of voice identity
processing consider the bases of perception as acoustic dimensions such as
fundamental frequency, harmonics-to-noise ratio, and formant dispersion.
However, these findings do not account for naturalistic speech and
within-speaker variability. Representational spaces of current self-supervised
models have shown significant performance in various speech-related tasks. In
this work, we demonstrate that self-supervised representations from different
families (e.g., generative, contrastive, and predictive models) are
significantly better for speaker identification over acoustic representations.
We also show that such a speaker identification task can be used to better
understand the nature of acoustic information representation in different
layers of these powerful networks. By evaluating speaker identification
accuracy across acoustic, phonemic, prosodic, and linguistic variants, we
report similarity between model performance and human identity perception. We
further examine these similarities by juxtaposing the encoding spaces of models
and humans and challenging the use of distance metrics as a proxy for speaker
proximity. Lastly, we show that some models can predict brain responses in
Auditory and Language regions during naturalistic stimuli.

自我监督表示与声音识别中的说话人身份有关，并能更好地理解强大网络中不同层次的声学信息表示，通过评估声学、语音、韵律和语言变体之间的说话人识别准确性，对比模型和人类的编码空间相似性，旨在挑战距离度量作为说话人接近程度的代理，并展示某些模型能够预测自然刺激下听觉和语言区域的脑部反应。

自监督模型与人类中的演讲者身份编码评估

Evaluating Speaker Identity Coding in Self-supervised Models and Humans

Accurately predicting heart activity and other biological signals is crucial
for diagnosis and monitoring. Given that speech is an outcome of multiple
physiological systems, a significant body of work studied the acoustic
correlates of heart activity. Recently, self-supervised models have excelled in
speech-related tasks compared to traditional acoustic methods. However, the
robustness of data-driven representations in predicting heart activity remained
unexplored. In this study, we demonstrate that self-supervised speech models
outperform acoustic features in predicting heart activity parameters. We also
emphasize the impact of individual variability on model generalizability. These
findings underscore the value of data-driven representations in such tasks and
the need for more speech-based physiological data to mitigate speaker-related
challenges.

通过研究语音与心脏活动的声学相关性，本研究证明了自监督语音模型在预测心脏活动参数方面胜过传统声学特征，并强调了个体变异对模型的普适性的影响，这些发现突显了数据驱动模型在这类任务中的价值，并需要更多基于语音的生理数据来解决与说话者相关的挑战。