We introduce a bilingual solution to support English as secondary locale for
most primary locales in hybrid automatic speech recognition (ASR) settings. Our
key developments constitute: (a) pronunciation lexicon with grapheme units
instead of phone units, (b) a fully bilingual alignment model and subsequently
bilingual streaming transformer model, (c) a parallel encoder structure with
language identification (LID) loss, (d) parallel encoder with an auxiliary loss
for monolingual projections. We conclude that in comparison to LID loss, our
proposed auxiliary loss is superior in specializing the parallel encoders to
respective monolingual locales, and that contributes to stronger bilingual
learning. We evaluate our work on large-scale training and test tasks for
bilingual Spanish (ES) and bilingual Italian (IT) applications. Our bilingual
models demonstrate strong English code-mixing capability. In particular, the
bilingual IT model improves the word error rate (WER) for a code-mix IT task
from 46.5% to 13.8%, while also achieving a close parity (9.6%) with the
monolingual IT model (9.5%) over IT tests.

我们介绍了一种支持英语作为主要语境的混合自动语音识别中的英语为辅助语境的双语解决方案，通过使用字素单元而不是音素单元的发音词典、完全双语对齐模型以及双语流转换模型、具有语种识别损失的并行编码器结构以及辅助损失的并行编码器，我们证明了辅助损失相比于语种识别损失在使并行编码器专门化到各自的单语语境方面更为优越，并且这有助于更强的双语学习。我们针对双语西班牙语（ES）和双语意大利语（IT）应用进行了大规模训练和测试任务的评估。我们的双语模型展示了良好的混合使用英语能力。特别是，在混合使用意大利语任务中，双语意大利语模型将词错误率（WER）从 46.5% 降低到 13.8%，同时在意大利语测试中也实现了与单语意大利语模型（9.5%）接近的匹配度（9.6%）。

以字音元和辅助单语损失的双语流式自动语音识别

Bilingual Streaming ASR with Grapheme units and Auxiliary Monolingual  Loss

In a hybrid automatic speech recognition (ASR) system, a pronunciation
lexicon (PL) and a language model (LM) are essential to correctly retrieve
spoken word sequences. Being a morphologically complex language, the vocabulary
of Malayalam is so huge and it is impossible to build a PL and an LM that cover
all diverse word forms. Usage of subword tokens to build PL and LM, and
combining them to form words after decoding, enables the recovery of many out
of vocabulary words. In this work we investigate the impact of using syllables
as subword tokens instead of words in Malayalam ASR, and evaluate the relative
improvement in lexicon size, model memory requirement and word error rate.

本文探讨了在马拉雅拉姆语自动语音识别中使用音节子单元作为音素字典和语言模型，并评估了词汇量、内存需求和词语错误率的相对改善。

使用音节子词标记进行开放词汇马拉雅拉姆语语音识别

Syllable Subword Tokens for Open Vocabulary Speech Recognition in Malayalam

As dialogue systems are becoming more and more interactional and social, also
the accurate automatic speech recognition (ASR) of conversational speech is of
increasing importance. This shifts the focus from short, spontaneous,
task-oriented dialogues to the much higher complexity of casual face-to-face
conversations. However, the collection and annotation of such conversations is
a time-consuming process and data is sparse for this specific speaking style.
This paper presents ASR experiments with read and conversational Austrian
German as target. In order to deal with having only limited resources available
for conversational German and, at the same time, with a large variation among
speakers with respect to pronunciation characteristics, we improve a
Kaldi-based ASR system by incorporating a (large) knowledge-based pronunciation
lexicon, while exploring different data-based methods to restrict the number of
pronunciation variants for each lexical entry. We achieve best WER of 0.4% on
Austrian German read speech and best average WER of 48.5% on conversational
speech. We find that by using our best pronunciation lexicon a similarly high
performance can be achieved than by increasing the size of the data used for
the language model by approx. 360% to 760%. Our findings indicate that for
low-resource scenarios -- despite the general trend in speech technology
towards using data-based methods only -- knowledge-based approaches are a
successful, efficient method.

本文研究使用基于知识的发音词典来提高德语对话语音识别的性能。实验结果表明，相对于增加语言模型数据的大小，使用发音词典可以在低资源语料情况下取得高性能。