Deep neural networks, such as the Deep-FSMN, have been widely studied for
keyword spotting (KWS) applications while suffering expensive computation and
storage. Therefore, network compression technologies like binarization are
studied to deploy KWS models on edge. In this paper, we present a strong yet
efficient binary neural network for KWS, namely BiFSMNv2, pushing it to the
real-network accuracy performance. First, we present a Dual-scale Thinnable
1-bit-Architecture to recover the representation capability of the binarized
computation units by dual-scale activation binarization and liberate the
speedup potential from an overall architecture perspective. Second, we also
construct a Frequency Independent Distillation scheme for KWS
binarization-aware training, which distills the high and low-frequency
components independently to mitigate the information mismatch between
full-precision and binarized representations. Moreover, we propose the Learning
Propagation Binarizer, a general and efficient binarizer that enables the
forward and backward propagation of binary KWS networks to be continuously
improved through learning. We implement and deploy the BiFSMNv2 on ARMv8
real-world hardware with a novel Fast Bitwise Computation Kernel, which is
proposed to fully utilize registers and increase instruction throughput.
Comprehensive experiments show our BiFSMNv2 outperforms existing binary
networks for KWS by convincing margins across different datasets and achieves
comparable accuracy with the full-precision networks (only a tiny 1.51% drop on
Speech Commands V1-12). We highlight that benefiting from the compact
architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive
25.1x speedup and 20.2x storage-saving on edge hardware.

本文介绍了一种强大而高效的二进制神经网络 BiFSMNv2，用于边缘端的关键词识别应用中。使用双尺度可调节的 1 位体系结构，独立分离高低频组分以缓解精度和二进制之间的信息不匹配，并提出了学习传播二元化算子，用于二进制网络的前向和后向传播的连续完善。与现有二进制网络相比，BiFSMNv2 在不同数据集上的性能优异，边缘硬件上具有高达 25.1 倍的加速和 20.2 倍的存储节省。