Low-power event-driven computation and inherent temporal dynamics render
spiking neural networks (SNNs) ideal candidates for processing highly dynamic
and asynchronous signals from event-based sensors. However, due to the
challenges in training and architectural design constraints, there is a
scarcity of competitive demonstrations of SNNs in event-based dense prediction
compared to artificial neural networks (ANNs). In this work, we construct an
efficient spiking encoder-decoder network for large-scale event-based semantic
segmentation tasks, optimizing the encoder with hierarchical search. To improve
learning from highly dynamic event streams, we exploit the intrinsic adaptive
threshold of spiking neurons to modulate network activation. Additionally, we
develop a dual-path spiking spatially-adaptive modulation (SSAM) block to
enhance the representation of sparse events, significantly improving network
performance. Our network achieves 72.57% mean intersection over union (MIoU) on
the DDD17 dataset and 57.22% MIoU on the newly proposed larger DSEC-Semantic
dataset, surpassing current record ANNs by 4% while utilizing much lower
computation costs. To the best of our knowledge, this is the first instance of
SNNs outperforming ANNs in challenging event-based semantic segmentation tasks,
demonstrating their immense potential in event-based vision. Our code will be
publicly available.

本文提出了一种高效的 SNN 编码器 - 解码器网络，并结合层次搜索来优化编码器，从而实现对大规模事件信号的语义分割任务。此外，利用神经元固有的自适应阈值来调节网络激活，并开发了双通道刺激的空间自适应调制 (SSAM) 块，显著提高了网络性能，从而使得 SNN 在事件驱动的视觉任务上的表现优于 ANN 达到 72.57% 和 57.22% 的平均交并比 (MIoU)，并且具有更低的计算成本。