In this work, we propose a new parameter-efficient learning framework based
on neural model reprogramming for cross-lingual speech recognition, which can
\textbf{re-purpose} well-trained English automatic speech recognition (ASR)
models to recognize the other languages. We design different auxiliary neural
architectures focusing on learnable pre-trained feature enhancement that, for
the first time, empowers model reprogramming on ASR. Specifically, we
investigate how to select trainable components (i.e., encoder) of a
conformer-based RNN-Transducer, as a frozen pre-trained backbone. Experiments
on a seven-language multilingual LibriSpeech speech (MLS) task show that model
reprogramming only requires 4.2% (11M out of 270M) to 6.8% (45M out of 660M) of
its original trainable parameters from a full ASR model to perform competitive
results in a range of 11.9% to 8.1% WER averaged across different languages. In
addition, we discover different setups to make large-scale pre-trained ASR
succeed in both monolingual and multilingual speech recognition. Our methods
outperform existing ASR tuning architectures and their extension with
self-supervised losses (e.g., w2v-bert) in terms of lower WER and better
training efficiency.

使用神经模型重编程的参数高效学习框架，可在跨语音识别中重新利用训练良好的英文自动语音识别模型，实现大规模预训练 ASR 成功，提高了训练效率。

从英语到更多语言：参数高效的模型重编程用于跨语言语音识别

From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition

Transfer learning (TL) approaches have shown promising results when handling
tasks with limited training data. However, considerable memory and
computational resources are often required for fine-tuning pre-trained neural
networks with target domain data. In this work, we introduce a novel method for
leveraging pre-trained models for low-resource (music) classification based on
the concept of Neural Model Reprogramming (NMR). NMR aims at re-purposing a
pre-trained model from a source domain to a target domain by modifying the
input of a frozen pre-trained model. In addition to the known,
input-independent, reprogramming method, we propose an advanced reprogramming
paradigm: Input-dependent NMR, to increase adaptability to complex input data
such as musical audio. Experimental results suggest that a neural model
pre-trained on large-scale datasets can successfully perform music genre
classification by using this reprogramming method. The two proposed
Input-dependent NMR TL methods outperform fine-tuning-based TL methods on a
small genre classification dataset.

本文提出使用神经模型重编程（NMR）对预训练模型进行修改，适应音乐分类等输入复杂的低资源任务。实验结果表明，使用 NMR 的迁移学习方法比微调方法更适用于小型音乐风格分类数据集。