Sequence-to-sequence (seq2seq) approach for low-resource ASR is a relatively
new direction in speech research. The approach benefits by performing model
training without using lexicon and alignments. However, this poses a new
problem of requiring more data compared to conventional DNN-HMM systems. In
this work, we attempt to use data from 10 BABEL languages to build a
multi-lingual seq2seq model as a prior model, and then port them towards 4
other BABEL languages using transfer learning approach. We also explore
different architectures for improving the prior multilingual seq2seq model. The
paper also discusses the effect of integrating a recurrent neural network
language model (RNNLM) with a seq2seq model during decoding. Experimental
results show that the transfer learning approach from the multilingual model
shows substantial gains over monolingual models across all 4 BABEL languages.
Incorporating an RNNLM also brings significant improvements in terms of %WER,
and achieves recognition performance comparable to the models trained with
twice more training data.

本文提出使用多语言数据建立先前模型，结合迁移学习的方法，通过在 10 个 BABEL 语言中训练 seq2seq 模型来改进音频识别，同时在解码时加入循环神经网络语言模型 (RNNLM) 来进一步提高模型的表现，实验结果表明，将先前的多语言模型应用于 4 种其他 BABEL 语言可以有效提高识别的准确度，并且加入 RNNLM 辅助也可以带来显著的性能提升。

多语言序列到序列语音识别：架构，转移学习和语言建模

Multilingual sequence-to-sequence speech recognition: architecture,  transfer learning, and language modeling

We describe Microsoft's conversational speech recognition system, in which we
combine recent developments in neural-network-based acoustic and language
modeling to advance the state of the art on the Switchboard recognition task.
Inspired by machine learning ensemble techniques, the system uses a range of
convolutional and recurrent neural networks. I-vector modeling and lattice-free
MMI training provide significant gains for all acoustic model architectures.
Language model rescoring with multiple forward and backward running RNNLMs, and
word posterior-based system combination provide a 20% boost. The best single
system uses a ResNet architecture acoustic model with RNNLM rescoring, and
achieves a word error rate of 6.9% on the NIST 2000 Switchboard task. The
combined system has an error rate of 6.2%, representing an improvement over
previously reported results on this benchmark task.

本文介绍 Microsoft 的会话式语音识别系统，其中采用了近期发展的基于神经网络的声学和语言建模技术，结合机器学习集成技术，使用了多种卷积和循环神经网络，通过 i-vector 模型建模和不带晶格的 MMI 训练，在声学建模架构的所有方面都提供了显著的增益率，使用多个前向和后向运行的 RNNLM 进行语言模型重新加权以及基于词后概率的系统组合，提高了 20％. 最佳单一系统使用 ResNet 架构声学模型与 RNNLM 重新加权，可在 NIST 2000 Switchboard 任务上实现 6.9％的词错误率。 结合系统的误差率为 6.2％，这是对此基准测试中以前报告的结果的改进。

Microsoft 2016 会话式语音识别系统

The Microsoft 2016 Conversational Speech Recognition System

We present a new tool for training neural network language models (NNLMs),
scoring sentences, and generating text. The tool has been written using Python
library Theano, which allows researcher to easily extend it and tune any aspect
of the training process. Regardless of the flexibility, Theano is able to
generate extremely fast native code that can utilize a GPU or multiple CPU
cores in order to parallelize the heavy numerical computations. The tool has
been evaluated in difficult Finnish and English conversational speech
recognition tasks, and significant improvement was obtained over our best
back-off n-gram models. The results that we obtained in the Finnish task were
compared to those from existing RNNLM and RWTHLM toolkits, and found to be as
good or better, while training times were an order of magnitude shorter.

使用 Python 库 Theano 开发而成的一款新的神经网络语言模型训练工具，在芬兰和英语会话语音识别任务中被评估，相对最佳的 N-gram 模型取得显著的改进；芬兰任务的结果表明，与现有的 RNNLM 和 RWTHLM 工具包相比，同等或更好，而训练时间则缩短了一个数量级。