In this work, we present Slimmable Neural Networks applied to the problem of
small-footprint keyword spotting. We show that slimmable neural networks allow
us to create super-nets from Convolutioanl Neural Networks and Transformers,
from which sub-networks of different sizes can be extracted. We demonstrate the
usefulness of these models on in-house Alexa data and Google Speech Commands,
and focus our efforts on models for the on-device use case, limiting ourselves
to less than 250k parameters. We show that slimmable models can match (and in
some cases, outperform) models trained from scratch. Slimmable neural networks
are therefore a class of models particularly useful when the same functionality
is to be replicated at different memory and compute budgets, with different
accuracy requirements.

本文介绍了应用于小尺寸关键词检测问题的可调整神经网络。我们展示了可调整神经网络如何允许我们从卷积神经网络和 Transformer 创建超网络，从中可以提取不同尺寸的子网络。我们在本文中展示了这些模型的实用性，并重点关注设备上的用例，将自己限制在不到 250k 参数的范围内。我们展示了可调整模型可以匹配（甚至在某些情况下超过）从头开始训练的模型。因此，可调整神经网络是一类在不同的存储和计算预算以及不同的精度要求下复制相同功能特别有用的模型。