Millimeter wave (mmWave) based speech recognition provides more possibility
for audio-related applications, such as conference speech transcription and
eavesdropping. However, considering the practicality in real scenarios, latency
and recognizable vocabulary size are two critical factors that cannot be
overlooked. In this paper, we propose Radio2Text, the first mmWave-based system
for streaming automatic speech recognition (ASR) with a vocabulary size
exceeding 13,000 words. Radio2Text is based on a tailored streaming Transformer
that is capable of effectively learning representations of speech-related
features, paving the way for streaming ASR with a large vocabulary. To
alleviate the deficiency of streaming networks unable to access entire future
inputs, we propose the Guidance Initialization that facilitates the transfer of
feature knowledge related to the global context from the non-streaming
Transformer to the tailored streaming Transformer through weight inheritance.
Further, we propose a cross-modal structure based on knowledge distillation
(KD), named cross-modal KD, to mitigate the negative effect of low quality
mmWave signals on recognition performance. In the cross-modal KD, the audio
streaming Transformer provides feature and response guidance that inherit
fruitful and accurate speech information to supervise the training of the
tailored radio streaming Transformer. The experimental results show that our
Radio2Text can achieve a character error rate of 5.7% and a word error rate of
9.4% for the recognition of a vocabulary consisting of over 13,000 words.

本文提出了 Radio2Text，这是第一个基于毫米波的流式自动语音识别系统，具有超过 13,000 个词汇大小的识别能力。Radio2Text 基于定制的流式 Transformer，有效地学习了与语音相关的特征表示，为具有大词汇量的流式自动语音识别铺平了道路。实验结果表明，Radio2Text 能够对由超过 13,000 个词汇组成的输入实现 5.7% 的字符错误率和 9.4% 的词错误率。