Spiking Neural Networks (SNNs) can offer ultra low power/ energy consumption
for machine learning-based applications due to their sparse spike-based
operations. Currently, most of the SNN architectures need a significantly
larger model size to achieve higher accuracy, which is not suitable for
resource-constrained embedded applications. Therefore, developing SNNs that can
achieve high accuracy with acceptable memory footprint is highly needed. Toward
this, we propose a novel methodology that improves the accuracy of SNNs through
kernel size scaling. Its key steps include investigating the impact of
different kernel sizes on the accuracy, devising new sets of kernel sizes,
generating SNN architectures based on the selected kernel sizes, and analyzing
the accuracy-memory trade-offs for SNN model selection. The experimental
results show that our methodology achieves higher accuracy than
state-of-the-art (93.24% accuracy for CIFAR10 and 70.84% accuracy for CIFAR100)
with less than 10M parameters and up to 3.45x speed-up of searching time,
thereby making it suitable for embedded applications.

提出一种通过核尺寸缩放改善脉冲神经网络（SNNs）准确性的新方法，实现了比现有技术更高的准确性（对于 CIFAR10 为 93.24%，对于 CIFAR100 为 70.84%），同时具有小于 10M 参数和多达 3.45 倍的搜索时间加速，适用于嵌入式应用。

通过核尺寸缩放提高嵌入式脉冲神经网络的准确性的方法论

A Methodology for Improving Accuracy of Embedded Spiking Neural Networks  through Kernel Size Scaling

Recently, some large kernel convnets strike back with appealing performance
and efficiency. However, given the square complexity of convolution, scaling up
kernels can bring about an enormous amount of parameters and the proliferated
parameters can induce severe optimization problem. Due to these issues, current
CNNs compromise to scale up to 51x51 in the form of stripe convolution (i.e.,
51x5 + 5x51) and start to saturate as the kernel size continues growing. In
this paper, we delve into addressing these vital issues and explore whether we
can continue scaling up kernels for more performance gains. Inspired by human
vision, we propose a human-like peripheral convolution that efficiently reduces
over 90% parameter count of dense grid convolution through parameter sharing,
and manage to scale up kernel size to extremely large. Our peripheral
convolution behaves highly similar to human, reducing the complexity of
convolution from O(K^2) to O(logK) without backfiring performance. Built on
this, we propose Parameter-efficient Large Kernel Network (PeLK). Our PeLK
outperforms modern vision Transformers and ConvNet architectures like Swin,
ConvNeXt, RepLKNet and SLaK on various vision tasks including ImageNet
classification, semantic segmentation on ADE20K and object detection on MS
COCO. For the first time, we successfully scale up the kernel size of CNNs to
an unprecedented 101x101 and demonstrate consistent improvements.

我们提出了一种人类视觉启发的周边卷积方法，通过参数共享高效地减少了密集网格卷积的参数数量超过 90%，并且成功实现了卷积核大小的可扩展性。基于这种方法，我们提出了一种参数高效的大核心网络（PeLK），在各种视觉任务中，包括 ImageNet 分类、ADE20K 的语义分割和 MS COCO 的目标检测，PeLK 都优于现代视觉 Transformer 和 ConvNet 架构。我们首次将 CNN 的卷积核大小扩展到了 101x101，并展示了持续的改进效果。