Low resource automatic speech recognition (ASR) is a useful but thorny task, since deep learning ASR models usually need huge amounts of training data. The existing models mostly established a bottleneck (BN) layer by pre-training on a large source language, and transferring to the low resource target language. In this work, we introduced an adaptive activation network to the upper layers of ASR model, and applied different activation functions to different languages. We also proposed two approaches to train the model: (1) cross-lingual learning, replacing the activation function from source language to target language, (2) multilingual learning, jointly training the Connectionist Temporal Classification (CTC) loss of each language and the relevance of different languages. Our experiments on IARPA Babel datasets demonstrated that our approaches outperform the from-scratch training and traditional bottleneck feature based methods. In addition, combining the cross-lingual learning and multilingual learning together could further improve the performance of multilingual speech recognition.

本篇论文提出了一种适应性激活网络，用于深度学习ASR模型的上层，并将不同的激活函数应用于不同的语言，通过交叉语言学习和多语言学习优化模型，达到了在IARPA Babel数据集上超越传统的基于瓶颈特征和从头训练两种方法的效果提升，结合交叉语言学习和多语言学习可以进一步提高多语言语音识别的性能。

面向低资源多语言语音识别的自适应激活网络