While Connectionist Temporal Classification (CTC) models deliver
state-of-the-art accuracy in automated speech recognition (ASR) pipelines,
their performance has been limited by CPU-based beam search decoding. We
introduce a GPU-accelerated Weighted Finite State Transducer (WFST) beam search
decoder compatible with current CTC models. It increases pipeline throughput
and decreases latency, supports streaming inference, and also supports advanced
features like utterance-specific word boosting via on-the-fly composition. We
provide pre-built DLPack-based python bindings for ease of use with
Python-based machine learning frameworks at
this https URL We evaluated our decoder
for offline and online scenarios, demonstrating that it is the fastest beam
search decoder for CTC models. In the offline scenario it achieves up to 7
times more throughput than the current state-of-the-art CPU decoder and in the
online streaming scenario, it achieves nearly 8 times lower latency, with same
or better word error rate.

通过引入支持当前 CTC 模型的 GPU 加速加权有限状态转录器（WFST）波束搜索解码器，我们提供了一种 GPU 加速的波束搜索解码器，可以提高管道吞吐量，减少延迟，并支持流式推理，同时支持实时组合进行话语特定的单词增强。我们针对离线和在线场景评估了我们的解码器，证明它是 CTC 模型的最快波束搜索解码器。在离线场景中，它的吞吐量比当前最先进的 CPU 解码器高出多达 7 倍，在在线流式场景中，它的延迟几乎比之前更低，而且单词错误率相同或更好。