This paper introduces a novel approach for streaming openvocabulary keyword
spotting (KWS) with text-based keyword enrollment. For every input frame, the
proposed method finds the optimal alignment ending at the frame using
connectionist temporal classification (CTC) and aggregates the frame-level
acoustic embedding (AE) to obtain higher-level (i.e., character, word, or
phrase) AE that aligns with the text embedding (TE) of the target keyword text.
After that, we calculate the similarity of the aggregated AE and the TE. To the
best of our knowledge, this is the first attempt to dynamically align the audio
and the keyword text on-the-fly to attain the joint audio-text embedding for
KWS. Despite operating in a streaming fashion, our approach achieves
competitive performance on the LibriPhrase dataset compared to the
non-streaming methods with a mere 155K model parameters and a decoding
algorithm with time complexity O(U), where U is the length of the target
keyword at inference time.

本文介绍了一种新颖的流式开放性词汇关键词定位方法，通过基于文本的关键词注册，在每个输入帧上使用连接主义时间分类（CTC）寻找以该帧结尾的最佳对齐，并聚合帧级声学嵌入（AE）以获得与目标关键词文本的文本嵌入（TE）对齐的更高级别（即字符、词或短语）的 AE，然后计算聚合 AE 与 TE 的相似度，取得了流媒体开放性词汇关键词定位的联合音频文本嵌入的最佳效果，尽管以流式方式操作，但我们的方法在 LibriPhrase 数据集上取得了竞争性的性能，仅使用了 155K 个模型参数和时间复杂度为 O (U) 的解码算法，其中 U 是推断时目标关键词的长度。