Spiking Neural Networks (SNNs) can offer ultra low power/ energy consumption
for machine learning-based applications due to their sparse spike-based
operations. Currently, most of the SNN architectures need a significantly
larger model size to achieve higher accuracy, which is not suitable for
resource-constrained embedded applications. Therefore, developing SNNs that can
achieve high accuracy with acceptable memory footprint is highly needed. Toward
this, we propose a novel methodology that improves the accuracy of SNNs through
kernel size scaling. Its key steps include investigating the impact of
different kernel sizes on the accuracy, devising new sets of kernel sizes,
generating SNN architectures based on the selected kernel sizes, and analyzing
the accuracy-memory trade-offs for SNN model selection. The experimental
results show that our methodology achieves higher accuracy than
state-of-the-art (93.24% accuracy for CIFAR10 and 70.84% accuracy for CIFAR100)
with less than 10M parameters and up to 3.45x speed-up of searching time,
thereby making it suitable for embedded applications.

提出一种通过核尺寸缩放改善脉冲神经网络（SNNs）准确性的新方法，实现了比现有技术更高的准确性（对于 CIFAR10 为 93.24%，对于 CIFAR100 为 70.84%），同时具有小于 10M 参数和多达 3.45 倍的搜索时间加速，适用于嵌入式应用。

通过核尺寸缩放提高嵌入式脉冲神经网络的准确性的方法论

A Methodology for Improving Accuracy of Embedded Spiking Neural Networks  through Kernel Size Scaling

Larger Spiking Neural Network (SNN) models are typically favorable as they
can offer higher accuracy. However, employing such models on the resource- and
energy-constrained embedded platforms is inefficient. Towards this, we present
a tinySNN framework that optimizes the memory and energy requirements of SNN
processing in both the training and inference phases, while keeping the
accuracy high. It is achieved by reducing the SNN operations, improving the
learning quality, quantizing the SNN parameters, and selecting the appropriate
SNN model. Furthermore, our tinySNN quantizes different SNN parameters (i.e.,
weights and neuron parameters) to maximize the compression while exploring
different combinations of quantization schemes, precision levels, and rounding
schemes to find the model that provides acceptable accuracy. The experimental
results demonstrate that our tinySNN significantly reduces the memory footprint
and the energy consumption of SNNs without accuracy loss as compared to the
baseline network. Therefore, our tinySNN effectively compresses the given SNN
model to achieve high accuracy in a memory- and energy-efficient manner, hence
enabling the employment of SNNs for the resource- and energy-constrained
embedded applications.

该论文提出了 tinySNN 框架，可通过减少操作、优化学习质量、量化参数和选择合适的模型来在保持准确性的前提下，优化 SNN 处理的内存和能源需求，并且通过不同的量化方案、精度级别和舍入方案探索不同的 SNN 参数来实现模型压缩，从而实现对于资源和能源受限的嵌入式应用的高精度高效压缩。

tinySNN：面向内存和能量高效的脉冲神经网络

tinySNN: Towards Memory- and Energy-Efficient Spiking Neural Networks

Low-bit deep neural networks (DNNs) become critical for embedded applications
due to their low storage requirement and computing efficiency. However, they
suffer much from the non-negligible accuracy drop. This paper proposes the
stochastic quantization (SQ) algorithm for learning accurate low-bit DNNs. The
motivation is due to the following observation. Existing training algorithms
approximate the real-valued elements/filters with low-bit representation all
together in each iteration. The quantization errors may be small for some
elements/filters, while are remarkable for others, which lead to inappropriate
gradient direction during training, and thus bring notable accuracy drop.
Instead, SQ quantizes a portion of elements/filters to low-bit with a
stochastic probability inversely proportional to the quantization error, while
keeping the other portion unchanged with full-precision. The quantized and
full-precision portions are updated with corresponding gradients separately in
each iteration. The SQ ratio is gradually increased until the whole network is
quantized. This procedure can greatly compensate the quantization error and
thus yield better accuracy for low-bit DNNs. Experiments show that SQ can
consistently and significantly improve the accuracy for different low-bit DNNs
on various datasets and various network structures.

本文介绍了一种用于学习准确的低比特深度神经网络的随机量化算法，其逐渐将网络量化，并可显著提高在不同数据集和结构上的低比特深度神经网络的准确度，适用于嵌入式应用。