Weight quantization is used to deploy high-performance deep learning models
on resource-limited hardware, enabling the use of low-precision integers for
storage and computation. Spiking neural networks (SNNs) share the goal of
enhancing efficiency, but adopt an 'event-driven' approach to reduce the power
consumption of neural network inference. While extensive research has focused
on weight quantization, quantization-aware training (QAT), and their
application to SNNs, the precision reduction of state variables during training
has been largely overlooked, potentially diminishing inference performance.
This paper introduces two QAT schemes for stateful neurons: (i) a uniform
quantization strategy, an established method for weight quantization, and (ii)
threshold-centered quantization, which allocates exponentially more
quantization levels near the firing threshold. Our results show that increasing
the density of quantization levels around the firing threshold improves
accuracy across several benchmark datasets. We provide an ablation analysis of
the effects of weight and state quantization, both individually and combined,
and how they impact models. Our comprehensive empirical evaluation includes
full precision, 8-bit, 4-bit, and 2-bit quantized SNNs, using QAT, stateful QAT
(SQUAT), and post-training quantization methods. The findings indicate that the
combination of QAT and SQUAT enhance performance the most, but given the choice
of one or the other, QAT improves performance by the larger degree. These
trends are consistent all datasets. Our methods have been made available in our
Python library snnTorch: this https URL

使用权重量化和量化感知训练 (QAT) 技术，研究了在功耗较低的神经网络中，通过为状态变量进行量化以提高推断性能的效果，并发现将量化级别密集分布在阈值附近可以提高准确性。在多个数据集上进行的实验评估表明，QAT 和状态量化训练 (SQUAT) 相结合能最大程度提高性能，而单独使用 QAT 效果最明显，这些趋势一致性地出现在所有数据集上。