Previous work has established that a person's demographics and speech style
affect how well speech processing models perform for them. But where does this
bias come from? In this work, we present the Speech Embedding Association Test
(SpEAT), a method for detecting bias in one type of model used for many speech
tasks: pre-trained models. The SpEAT is inspired by word embedding association
tests in natural language processing, which quantify intrinsic bias in a
model's representations of different concepts, such as race or valence
(something's pleasantness or unpleasantness) and capture the extent to which a
model trained on large-scale socio-cultural data has learned human-like biases.
Using the SpEAT, we test for six types of bias in 16 English speech models
(including 4 models also trained on multilingual data), which come from the
wav2vec 2.0, HuBERT, WavLM, and Whisper model families. We find that 14 or more
models reveal positive valence (pleasantness) associations with abled people
over disabled people, with European-Americans over African-Americans, with
females over males, with U.S. accented speakers over non-U.S. accented
speakers, and with younger people over older people. Beyond establishing that
pre-trained speech models contain these biases, we also show that they can have
real world effects. We compare biases found in pre-trained models to biases in
downstream models adapted to the task of Speech Emotion Recognition (SER) and
find that in 66 of the 96 tests performed (69%), the group that is more
associated with positive valence as indicated by the SpEAT also tends to be
predicted as speaking with higher valence by the downstream model. Our work
provides evidence that, like text and image-based models, pre-trained speech
based-models frequently learn human-like biases. Our work also shows that bias
found in pre-trained models can propagate to the downstream task of SER.

我们提出了一种名为 SpEAT 的方法，用于检测预训练模型中的偏见；使用该方法对 16 个英语语音模型进行测试，发现预训练语音模型普遍存在人类类似的偏见，并证明这些偏见会对 Speech Emotion Recognition 任务产生实际影响。

预训练的语音处理模型含有类人的偏见并传播至语音情感识别

Pre-trained Speech Processing Models Contain Human-Like Biases that  Propagate to Speech Emotion Recognition

Memory constraint of always-on devices is one of the major concerns when
deploying speech processing models on these devices. While larger models
trained with sufficiently large amount of data generally perform better, making
them fit in the device memory is a demanding challenge. In this paper, we aim
to reduce model size by reparameterizing model weights across Transformer
encoder layers and assuming a special weight composition and structure. More
specifically, inspired by ResNet and the more recent LoRA work, we propose an
approach named ResidualTransformer, where each weight matrix in a Transformer
layer comprises 1) a shared full-rank component with its adjacent layers, and
2) a unique low-rank component to itself. The low-rank matrices only account
for a small amount of model size increase. In addition, we add diagonal weight
matrices to improve modeling capacity of the low-rank matrices. Experiments of
our 10k-hour speech recognition and speech translation tasks show that the
Transformer encoder size can be reduced by ~3X with very slight performance
degradation.

通过重新参数化模型权重，将 Transformer 编码器层中的模型权重缩小，可以降低模型大小，减轻设备内存的压力，并在语音处理任务上取得了显著的性能改进。