Event-based sensors are well suited for real-time processing due to their
fast response times and encoding of the sensory data as successive temporal
differences. These and other valuable properties, such as a high dynamic range,
are suppressed when the data is converted to a frame-based format. However,
most current methods either collapse events into frames or cannot scale up when
processing the event data directly event-by-event. In this work, we address the
key challenges of scaling up event-by-event modeling of the long event streams
emitted by such sensors, which is a particularly relevant problem for
neuromorphic computing. While prior methods can process up to a few thousand
time steps, our model, based on modern recurrent deep state-space models,
scales to event streams of millions of events for both training and
inference.We leverage their stable parameterization for learning long-range
dependencies, parallelizability along the sequence dimension, and their ability
to integrate asynchronous events effectively to scale them up to long event
streams.We further augment these with novel event-centric techniques enabling
our model to match or beat the state-of-the-art performance on several event
stream benchmarks. In the Spiking Speech Commands task, we improve
state-of-the-art by a large margin of 6.6% to 87.1%. On the DVS128-Gestures
dataset, we achieve competitive results without using frames or convolutional
neural networks. Our work demonstrates, for the first time, that it is possible
to use fully event-based processing with purely recurrent networks to achieve
state-of-the-art task performance in several event-based benchmarks.

本研究探讨了使用完全事件驱动的处理方法和基于现代循环深度状态空间模型的事件流建模来实现几个事件流基准测试的最新性能。