An important and difficult task in code-switched speech recognition is to
recognize the language, as lots of words in two languages can sound similar,
especially in some accents. We focus on improving performance of end-to-end
Automatic Speech Recognition models by conditioning transformer layers on
language ID of words and character in the output in an per layer supervised
manner. To this end, we propose two methods of introducing language specific
parameters and explainability in the multi-head attention mechanism, and
implement a Temporal Loss that helps maintain continuity in input alignment.
Despite being unable to reduce WER significantly, our method shows promise in
predicting the correct language from just spoken data. We introduce
regularization in the language prediction by dropping LID in the sequence,
which helps align long repeated output sequences.

通过条件化变换器层上的语言 ID，我们提出了两种引入语言特定参数和可解释性以及实现辅助性的时间损失方法，以改进端到端自动语音识别模型的性能。尽管无法显著减少词错误率，但我们的方法在通过仅仅的口语数据预测正确语言方面表现出潜力。我们通过在序列中删除语言 ID 来引入语言预测的正则化，有助于对齐长重复的输出序列。