Multilingual automatic speech recognition (ASR) systems mostly benefit low
resource languages but suffer degradation in performance across several
languages relative to their monolingual counterparts. Limited studies have
focused on understanding the languages behaviour in the multilingual speech
recognition setups. In this paper, a novel data-driven approach is proposed to
investigate the cross-lingual acoustic-phonetic similarities. This technique
measures the similarities between posterior distributions from various
monolingual acoustic models against a target speech signal. Deep neural
networks are trained as mapping networks to transform the distributions from
different acoustic models into a directly comparable form. The analysis
observes that the languages closeness can not be truly estimated by the volume
of overlapping phonemes set. Entropy analysis of the proposed mapping networks
exhibits that a language with lesser overlap can be more amenable to
cross-lingual transfer, and hence more beneficial in the multilingual setup.
Finally, the proposed posterior transformation approach is leveraged to fuse
monolingual models for a target language. A relative improvement of ~8% over
monolingual counterpart is achieved.

本文提出了一种新的数据驱动方法来研究跨语音识别方案中的跨语言声学语音相似性，通过训练深度神经网络来将来自不同声音模型的分布转化为可直接比较的形式，并通过熵分析发现少重叠语音的语言更易于跨语言传输，在融合单语言模型方面取得了相对于单语言识别的 8％的改进。