Low resource automatic speech recognition (ASR) is a useful but thorny task,
since deep learning ASR models usually need huge amounts of training data. The
existing models mostly established a bottleneck (BN) layer by pre-training on a
large source language, and transferring to the low resource target language. In
this work, we introduced an adaptive activation network to the upper layers of
ASR model, and applied different activation functions to different languages.
We also proposed two approaches to train the model: (1) cross-lingual learning,
replacing the activation function from source language to target language, (2)
multilingual learning, jointly training the Connectionist Temporal
Classification (CTC) loss of each language and the relevance of different
languages. Our experiments on IARPA Babel datasets demonstrated that our
approaches outperform the from-scratch training and traditional bottleneck
feature based methods. In addition, combining the cross-lingual learning and
multilingual learning together could further improve the performance of
multilingual speech recognition.

本篇论文提出了一种适应性激活网络，用于深度学习 ASR 模型的上层，并将不同的激活函数应用于不同的语言，通过交叉语言学习和多语言学习优化模型，达到了在 IARPA Babel 数据集上超越传统的基于瓶颈特征和从头训练两种方法的效果提升，结合交叉语言学习和多语言学习可以进一步提高多语言语音识别的性能。