How to effectively and efficiently deal with spatio-temporal event streams,
where the events are generally sparse and non-uniform and have the microsecond
temporal resolution, is of great value and has various real-life applications.
Spiking neural network (SNN), as one of the brain-inspired event-triggered
computing models, has the potential to extract effective spatio-temporal
features from the event streams. However, when aggregating individual events
into frames with a new higher temporal resolution, existing SNN models do not
attach importance to that the serial frames have different signal-to-noise
ratios since event streams are sparse and non-uniform. This situation
interferes with the performance of existing SNNs. In this work, we propose a
temporal-wise attention SNN (TA-SNN) model to learn frame-based representation
for processing event streams. Concretely, we extend the attention concept to
temporal-wise input to judge the significance of frames for the final decision
at the training stage, and discard the irrelevant frames at the inference
stage. We demonstrate that TA-SNN models improve the accuracy of event streams
classification tasks. We also study the impact of multiple-scale temporal
resolutions for frame-based representation. Our approach is tested on three
different classification tasks: gesture recognition, image classification, and
spoken digit recognition. We report the state-of-the-art results on these
tasks, and get the essential improvement of accuracy (almost 19\%) for gesture
recognition with only 60 ms.

该论文提出了一种基于时间注意力的尖峰神经网络模型来处理事件流，能够有效地提取时空特征，提高分类准确率。