An important and difficult task in code-switched speech recognition is to
recognize the language, as lots of words in two languages can sound similar,
especially in some accents. We focus on improving performance of end-to-end
Automatic Speech Recognition models by conditioning transformer layers on
language ID of words and character in the output in an per layer supervised
manner. To this end, we propose two methods of introducing language specific
parameters and explainability in the multi-head attention mechanism, and
implement a Temporal Loss that helps maintain continuity in input alignment.
Despite being unable to reduce WER significantly, our method shows promise in
predicting the correct language from just spoken data. We introduce
regularization in the language prediction by dropping LID in the sequence,
which helps align long repeated output sequences.

通过条件化变换器层上的语言 ID，我们提出了两种引入语言特定参数和可解释性以及实现辅助性的时间损失方法，以改进端到端自动语音识别模型的性能。尽管无法显著减少词错误率，但我们的方法在通过仅仅的口语数据预测正确语言方面表现出潜力。我们通过在序列中删除语言 ID 来引入语言预测的正则化，有助于对齐长重复的输出序列。

使用语言模型的语言切换语音识别：古吉拉特英语

Gujarati-English Code-Switching Speech Recognition using ensemble  prediction of spoken language

Building a multilingual Automated Speech Recognition (ASR) system in a
linguistically diverse country like India can be a challenging task due to the
differences in scripts and the limited availability of speech data. This
problem can be solved by exploiting the fact that many of these languages are
phonetically similar. These languages can be converted into a Common Label Set
(CLS) by mapping similar sounds to common labels. In this paper, new approaches
are explored and compared to improve the performance of CLS based multilingual
ASR model. Specific language information is infused in the ASR model by giving
Language ID or using CLS to Native script converter on top of the CLS
Multilingual model. These methods give a significant improvement in Word Error
Rate (WER) compared to the CLS baseline. These methods are further tried on
out-of-distribution data to check their robustness.

研究了基于通用标签集合（CLS）的多语言 ASR 的性能提升，通过给 ASR 模型添加语言 ID 或使用 CLS 到本地脚本转换器，可以将特定的语言信息注入到 ASR 模型中。这些方法与基线模型相比在词错误率上得到了显着改善，并在分布数据上进行了有效性测试。