We present a streaming, Transformer-based end-to-end automatic speech recognition (ASR) architecture which achieves efficient neural inference through compute cost amortization. Our architecture creates sparse computation pathways dynamically at inference time, resulting in selective use of compute resources throughout decoding, enabling significant reductions in compute with minimal impact on accuracy. The fully differentiable architecture is trained end-to-end with an accompanying lightweight arbitrator mechanism operating at the frame-level to make dynamic decisions on each input while a tunable loss function is used to regularize the overall level of compute against predictive performance. We report empirical results from experiments using the compute amortized Transformer-Transducer (T-T) model conducted on LibriSpeech data. Our best model can achieve a 60% compute cost reduction with only a 3% relative word error rate (WER) increase.

该论文提出了一种基于Transformer的端到端自动语音识别体系结构，该体系结构通过计算成本分摊实现高效的神经推断，其在推断期间动态创建稀疏计算路径，实现计算资源的选择性使用，并使用可调节的损失函数规范预测性能对计算总体水平进行计算。在LibriSpeech数据上进行实验，最佳模型只增加3％的相对字错误率并降低60％的计算成本。

流式ASR的计算成本分摊Transformer