We present a compact spiking convolutional neural network (SCNN) and spiking
multilayer perceptron (SMLP) to recognize ten different gestures in dark and
bright light environments, using a $9.6 single-photon avalanche diode (SPAD)
array. In our hand gesture recognition (HGR) system, photon intensity data was
leveraged to train and test the network. A vanilla convolutional neural network
(CNN) was also implemented to compare the performance of SCNN with the same
network topologies and training strategies. Our SCNN was trained from scratch
instead of being converted from the CNN. We tested the three models in dark and
ambient light (AL)-corrupted environments. The results indicate that SCNN
achieves comparable accuracy (90.8%) to CNN (92.9%) and exhibits lower floating
operations with only 8 timesteps. SMLP also presents a trade-off between
computational workload and accuracy. The code and collected datasets of this
work are available at this https URL

我们提出了一种紧凑的脉冲卷积神经网络（SCNN）和脉冲多层感知机（SMLP），用于在黑暗和明亮的环境中使用 9.6 个单光子雪崩二极管（SPAD）阵列来识别十种不同的手势，在我们的手势识别系统中，利用光子强度数据来训练和测试网络，与同样的网络拓扑和训练策略下的卷积神经网络（CNN）进行比较，结果显示 SCNN 在黑暗和环境光污染环境中实现了可比较的准确率（90.8%），并且只有 8 个时间步长的浮点操作。SMLP 还呈现出计算工作量和准确性之间的平衡。

利用低成本单光子雪崩二极管阵列增强手势识别的脉冲神经网络

Spiking Neural Network Enhanced Hand Gesture Recognition Using Low-Cost  Single-photon Avalanche Diode Array

As intelligent systems become increasingly important in our daily lives, new
ways of interaction are needed. Classical user interfaces pose issues for the
physically impaired and are partially not practical or convenient. Gesture
recognition is an alternative, but often not reactive enough when conventional
cameras are used. This work proposes a Spiking Convolutional Neural Network,
processing event- and depth data for gesture recognition. The network is
simulated using the open-source neuromorphic computing framework LAVA for
offline training and evaluation on an embedded system. For the evaluation three
open source data sets are used. Since these do not represent the applied
bi-modality, a new data set with synchronized event- and depth data was
recorded. The results show the viability of temporal encoding on depth
information and modality fusion, even on differently encoded data, to be
beneficial to network performance and generalization capabilities.

智能系统在我们日常生活中变得越来越重要，需要新的交互方式。本文提出的脉冲卷积神经网络利用事件和深度数据进行手势识别，通过在嵌入式系统上进行离线训练和评估，并使用开源的神经形态计算框架 LAVA 进行模拟。研究结果表明，在深度信息和模态融合中，时间编码及不同编码的数据对网络性能和泛化能力有积极效果。