Event cameras offer high temporal resolution and dynamic range with minimal
motion blur, making them promising for object detection tasks. While Spiking
Neural Networks (SNNs) are a natural match for event-based sensory data and
enable ultra-energy efficient and low latency inference on neuromorphic
hardware, Artificial Neural Networks (ANNs) tend to display more stable
training dynamics and faster convergence resulting in greater task performance.
Hybrid SNN-ANN approaches are a promising alternative, enabling to leverage the
strengths of both SNN and ANN architectures. In this work, we introduce the
first Hybrid Attention-based SNN-ANN backbone for object detection using event
cameras. We propose a novel Attention-based SNN-ANN bridge module to capture
sparse spatial and temporal relations from the SNN layer and convert them into
dense feature maps for the ANN part of the backbone. Experimental results
demonstrate that our proposed method surpasses baseline hybrid and SNN-based
approaches by significant margins, with results comparable to existing
ANN-based methods. Extensive ablation studies confirm the effectiveness of our
proposed modules and architectural choices. These results pave the way toward a
hybrid SNN-ANN architecture that achieves ANN like performance at a drastically
reduced parameter budget. We implemented the SNN blocks on digital neuromorphic
hardware to investigate latency and power consumption and demonstrate the
feasibility of our approach.

我们介绍了一种基于事件相机的混合注意力脉冲神经网络 — 人工神经网络（Hybrid Attention-based SNN-ANN）骨干架构，用于目标检测。通过引入新颖的基于注意力的 SNN-ANN 桥接模块，我们能够从 SNN 层捕捉稀疏的空间和时间关系，并将其转换为 ANN 骨干的稠密特征图。实验结果表明，我们的方法在性能上超过了基线混合和基于 SNN 的方法，并与现有的基于 ANN 的方法相媲美。深入的消融研究确认了我们提出的模块和架构选择的有效性。这些结果为实现在大幅度减少参数预算的情况下达到 ANN 类似性能的混合 SNN-ANN 架构铺平了道路。我们在数字神经形态硬件上实现了 SNN 块，以研究延迟和功耗，并证明了我们的方法的可行性。