In this work, we devise a parameter-efficient solution to bring differential
privacy (DP) guarantees into adaptation of a cross-lingual speech classifier.
We investigate a new frozen pre-trained adaptation framework for DP-preserving
speech modeling without full model fine-tuning. First, we introduce a noisy
teacher-student ensemble into a conventional adaptation scheme leveraging a
frozen pre-trained acoustic model and attain superior performance than DP-based
stochastic gradient descent (DPSGD). Next, we insert residual adapters (RA)
between layers of the frozen pre-trained acoustic model. The RAs reduce
training cost and time significantly with a negligible performance drop.
Evaluated on the open-access Multilingual Spoken Words (MLSW) dataset, our
solution reduces the number of trainable parameters by 97.5% using the RAs with
only a 4% performance drop with respect to fine-tuning the cross-lingual speech
classifier while preserving DP guarantees.

通过噪声教师学生模型和残余适配器的引入，本研究提出了一种参数高效的解决方案，实现跨语言语音分类器的差分隐私保护，通过在训练之前进行一次性预训练，将模型的参数数量减少了 97.5%，并且只有 4% 的性能下降。