Using a vision-inspired keyword spotting framework, we propose an
architecture with input-dependent dynamic depth capable of processing streaming
audio. Specifically, we extend a conformer encoder with trainable binary gates
that allow us to dynamically skip network modules according to the input audio.
Our approach improves detection and localization accuracy on continuous speech
using Librispeech top-1000 most frequent words while maintaining a small memory
footprint. The inclusion of gates also reduces the average amount of processing
without affecting the overall performance. These benefits are shown to be even
more pronounced using the Google speech commands dataset placed over background
noise where up to 97% of the processing is skipped on non-speech inputs,
therefore making our method particularly interesting for an always-on keyword
spotter.

利用一种以视觉为灵感的关键词检测框架，我们提出了一种具有输入相关动态深度的架构，能够处理流媒体音频。我们通过在可训练的二进制门中扩展一个 conformer 编码器来动态跳过网络模块。我们的方法在使用 Librispeech 前 1000 个最常见单词进行持续语音上的检测和定位准确性方面有所提高，同时还保持了较小的内存占用。引入门也减少了处理的平均量，而不影响整体性能。在背景噪声下使用谷歌语音命令数据集时，这些好处尤为明显，非语音输入中可省略多达 97% 的处理，因此使我们的方法特别适用于始终开启的关键词检测器。

基于动态模块跳过的流式 conformer 编码器提升视觉驱动的关键词识别

Improving vision-inspired keyword spotting using dynamic module skipping  in streaming conformer encoder

Transformer-based end-to-end speech recognition has achieved great success.
However, the large footprint and computational overhead make it difficult to
deploy these models in some real-world applications. Model compression
techniques can reduce the model size and speed up inference, but the compressed
model has a fixed architecture which might be suboptimal. We propose a novel
Transformer encoder with Input-Dependent Dynamic Depth (I3D) to achieve strong
performance-efficiency trade-offs. With a similar number of layers at inference
time, I3D-based models outperform the vanilla Transformer and the static pruned
model via iterative layer pruning. We also present interesting analysis on the
gate probabilities and the input-dependency, which helps us better understand
deep encoders.

该研究提出了一种新的 Transformer 编码器模型，并利用输入依赖动态深度 (I3D) 实现了性能 - 效率的良好均衡，该方法可用于压缩模型大小并通过迭代层剪枝处理以提高模型性能，同时对门控概率和输入依赖性进行了分析以更好地理解深度编码器。