Streaming ASR with strict latency constraints is required in many speech
recognition applications. In order to achieve the required latency, streaming
ASR models sacrifice accuracy compared to non-streaming ASR models due to lack
of future input context. Previous research has shown that streaming and
non-streaming ASR for RNN Transducers can be unified by cascading causal and
non-causal encoders. This work improves upon this cascaded encoders framework
by leveraging two streaming non-causal encoders with variable input context
sizes that can produce outputs at different audio intervals (e.g. fast and
slow). We propose a novel parallel time-synchronous beam search algorithm for
transducers that decodes from fast-slow encoders, where the slow encoder
corrects the mistakes generated from the fast encoder. The proposed algorithm,
achieves up to 20% WER reduction with a slight increase in token emission
delays on the public Librispeech dataset and in-house datasets. We also explore
techniques to reduce the computation by distributing processing between the
fast and slow encoders. Lastly, we explore sharing the parameters in the fast
encoder to reduce the memory footprint. This enables low latency processing on
edge devices with low computation cost and a low memory footprint.

该研究提出了在流式自动语音识别中应用串联有因果和非因果编码器的方法，并使用两个有不同输入上下文大小和以不同的音频间隔产生输出的流式非因果编码器进行改进；同时提出了一种新的并行时间同步波束搜索算法。研究结果表明，在公共数据集 Librispeech 和内部数据集上，该算法在稍微增加音节延迟的情况下，实现 WER 降低 20％左右，并探索了分布式处理和共享参数降低内存占用的技术，以实现低计算成本和低内存占用的边缘设备上的低延迟处理。

流式并行传感器束搜索算法及快慢级联编码器

Streaming parallel transducer beam search with fast-slow cascaded encoders

End-to-end (E2E) models have shown to outperform state-of-the-art
conventional models for streaming speech recognition [1] across many
dimensions, including quality (as measured by word error rate (WER)) and
endpointer latency [2]. However, the model still tends to delay the predictions
towards the end and thus has much higher partial latency compared to a
conventional ASR model. To address this issue, we look at encouraging the E2E
model to emit words early, through an algorithm called FastEmit [3]. Naturally,
improving on latency results in a quality degradation. To address this, we
explore replacing the LSTM layers in the encoder of our E2E model with
Conformer layers [4], which has shown good improvements for ASR. Secondly, we
also explore running a 2nd-pass beam search to improve quality. In order to
ensure the 2nd-pass completes quickly, we explore non-causal Conformer layers
that feed into the same 1st-pass RNN-T decoder, an algorithm called Cascaded
Encoders [5]. Overall, we find that the Conformer RNN-T with Cascaded Encoders
offers a better quality and latency tradeoff for streaming ASR.

本文研究了使用不同算法，如 FastEmit，Conformer 层和 Cascaded Encoders 等，提高端到端模型在流式语音识别领域的质量和延迟平衡。