Recognizing written domain numeric utterances (e.g. I need $1.25.) can be
challenging for ASR systems, particularly when numeric sequences are not seen
during training. This out-of-vocabulary (OOV) issue is addressed in
conventional ASR systems by training part of the model on spoken domain
utterances (e.g. I need one dollar and twenty five cents.), for which numeric
sequences are composed of in-vocabulary numbers, and then using an FST
verbalizer to denormalize the result. Unfortunately, conventional ASR models
are not suitable for the low memory setting of on-device speech recognition.
E2E models such as RNN-T are attractive for on-device ASR, as they fold the AM,
PM and LM of a conventional model into one neural network. However, in the
on-device setting the large memory footprint of an FST denormer makes spoken
domain training more difficult. In this paper, we investigate techniques to
improve E2E model performance on numeric data. We find that using a
text-to-speech system to generate additional numeric training data, as well as
using a small-footprint neural network to perform spoken-to-written domain
denorming, yields improvement in several numeric classes. In the case of the
longest numeric sequences, we see reduction of WER by up to a factor of 8.

本文研究使用文本到语音系统和小型神经网络来解决在设备上语音识别中模型训练的问题，以提高 E2E 模型在数字数据上的性能表现。

优化端到端自动语音识别在数字序列上的性能

Improving Performance of End-to-End ASR on Numeric Sequences

The acoustic-to-word model based on the connectionist temporal classification
(CTC) criterion was shown as a natural end-to-end (E2E) model directly
targeting words as output units. However, the word-based CTC model suffers from
the out-of-vocabulary (OOV) issue as it can only model limited number of words
in the output layer and maps all the remaining words into an OOV output node.
Hence, such a word-based CTC model can only recognize the frequent words
modeled by the network output nodes. Our first attempt to improve the
acoustic-to-word model is a hybrid CTC model which consults a letter-based CTC
when the word-based CTC model emits OOV tokens during testing time. Then, we
propose a much better solution by training a mixed-unit CTC model which
decomposes all the OOV words into sequences of frequent words and multi-letter
units. Evaluated on a 3400 hours Microsoft Cortana voice assistant task, the
final acoustic-to-word solution improves the baseline word-based CTC by
relative 12.09% word error rate (WER) reduction when combined with our proposed
attention CTC. Such an E2E model without using any language model (LM) or
complex decoder outperforms the traditional context-dependent phoneme CTC which
has strong LM and decoder by relative 6.79%.

本论文研究了基于 CTC 准则的声音到单词模型的问题，并提出采用混合 CTC 模型和混合单元 CTC 模型的解决方案，最终实现了在没有使用语言模型或复杂解码器的情况下取得比传统 CD-CTC 模型更好的结果。