The training of automatic speech recognition (ASR) with little to no supervised data remains an open question. In this work, we demonstrate that streaming Transformer-Transducer (TT) models can be trained from scratch in consumer and accessible GPUs in their entirety with pseudo-labeled (PL) speech from foundational speech models (FSM). This allows training a robust ASR model just in one stage and does not require large data and computational budget compared to the two-step scenario with pre-training and fine-tuning. We perform a comprehensive ablation on different aspects of PL-based streaming TT models such as the impact of (1) shallow fusion of n-gram LMs, (2) contextual biasing with named entities, (3) chunk-wise decoding for low-latency streaming applications, and (4) TT overall performance as the function of the FSM size. Our results demonstrate that TT can be trained from scratch without supervised data, even with very noisy PLs. We validate the proposed framework on 6 languages from CommonVoice and propose multiple heuristics to filter out hallucinated PLs.

本研究解决了在缺乏监督数据条件下训练自动语音识别（ASR）模型的难题。我们提出了一种新方法，利用伪标记语音有效地从头开始训练流式变换器-转导器（TT）模型，结果表明这种方法能够在各种语言上获得强大的ASR性能，且不需要大量的数据和计算资源。

通过知识蒸馏与Whisper快速流式变换器自动语音识别原型开发