Large general-purpose transformer models have recently become the mainstay in
the realm of speech analysis. In particular, Whisper achieves state-of-the-art
results in relevant tasks such as speech recognition, translation, language
identification, and voice activity detection. However, Whisper models are not
designed to be used in real-time conditions, and this limitation makes them
unsuitable for a vast plethora of practical applications. In this paper, we
introduce Whispy, a system intended to bring live capabilities to the Whisper
pretrained models. As a result of a number of architectural optimisations,
Whispy is able to consume live audio streams and generate high level, coherent
voice transcriptions, while still maintaining a low computational cost. We
evaluate the performance of our system on a large repository of publicly
available speech datasets, investigating how the transcription mechanism
introduced by Whispy impacts on the Whisper output. Experimental results show
how Whispy excels in robustness, promptness, and accuracy.

Whispy 是一个能够实时处理语音流并生成高质量语音转录的系统，优化了 Whisper 预训练模型的架构，具备较低的计算成本，实验结果展示了其在鲁棒性、及时性和准确性方面的优势。