Event cameras offer high temporal resolution and dynamic range with minimal motion blur, making them promising for object detection tasks. While Spiking Neural Networks (SNNs) are a natural match for event-based sensory data and enable ultra-energy efficient and low latency inference on neuromorphic hardware, Artificial Neural Networks (ANNs) tend to display more stable training dynamics and faster convergence resulting in greater task performance. Hybrid SNN-ANN approaches are a promising alternative, enabling to leverage the strengths of both SNN and ANN architectures. In this work, we introduce the first Hybrid Attention-based SNN-ANN backbone for object detection using event cameras. We propose a novel Attention-based SNN-ANN bridge module to capture sparse spatial and temporal relations from the SNN layer and convert them into dense feature maps for the ANN part of the backbone. Experimental results demonstrate that our proposed method surpasses baseline hybrid and SNN-based approaches by significant margins, with results comparable to existing ANN-based methods. Extensive ablation studies confirm the effectiveness of our proposed modules and architectural choices. These results pave the way toward a hybrid SNN-ANN architecture that achieves ANN like performance at a drastically reduced parameter budget. We implemented the SNN blocks on digital neuromorphic hardware to investigate latency and power consumption and demonstrate the feasibility of our approach.

我们介绍了一种基于事件相机的混合注意力脉冲神经网络—人工神经网络（Hybrid Attention-based SNN-ANN）骨干架构，用于目标检测。通过引入新颖的基于注意力的SNN-ANN桥接模块，我们能够从SNN层捕捉稀疏的空间和时间关系，并将其转换为ANN骨干的稠密特征图。实验结果表明，我们的方法在性能上超过了基线混合和基于SNN的方法，并与现有的基于ANN的方法相媲美。深入的消融研究确认了我们提出的模块和架构选择的有效性。这些结果为实现在大幅度减少参数预算的情况下达到ANN类似性能的混合SNN-ANN架构铺平了道路。我们在数字神经形态硬件上实现了SNN块，以研究延迟和功耗，并证明了我们的方法的可行性。

基于空间和时间注意力的混合SNN-ANN网络用于基于事件的物体检测