Despite the success of neural networks in computer vision tasks, digital
'neurons' are a very loose approximation of biological neurons. Today's
learning approaches are designed to function on digital devices with digital
data representations such as image frames. In contrast, biological vision
systems are generally much more capable and efficient than state-of-the-art
digital computer vision algorithms. Event cameras are an emerging sensor
technology which imitates biological vision with asynchronously firing pixels,
eschewing the concept of the image frame. To leverage modern learning
techniques, many event-based algorithms are forced to accumulate events back to
image frames, somewhat squandering the advantages of event cameras.
We follow the opposite paradigm and develop a new type of neural network
which operates closer to the original event data stream. We demonstrate
state-of-the-art performance in angular velocity regression and competitive
optical flow estimation, while avoiding difficulties related to training SNN.
Furthermore, the processing latency of our proposed approached is less than
1/10 any other implementation, while continuous inference increases this
improvement by another order of magnitude.

利用事件相机的异步像素来开发新型神经网络在角速度回归和光流估计方面表现出最先进的性能。

EDeNN: 用于低延迟视觉的事件衰减神经网络

EDeNN: Event Decay Neural Networks for low latency vision

Spiking Neural Networks (SNNs) are bio-inspired networks that process
information conveyed as temporal spikes rather than numeric values. A spiking
neuron of an SNN only produces a spike whenever a significant number of spikes
occur within a short period of time. Due to their spike-based computational
model, SNNs can process output from event-based, asynchronous sensors without
any pre-processing at extremely lower power unlike standard artificial neural
networks. This is possible due to specialized neuromorphic hardware that
implements the highly-parallelizable concept of SNNs in silicon. Yet, SNNs have
not enjoyed the same rise of popularity as artificial neural networks. This not
only stems from the fact that their input format is rather unconventional but
also due to the challenges in training spiking networks. Despite their temporal
nature and recent algorithmic advances, they have been mostly evaluated on
classification problems. We propose, for the first time, a temporal regression
problem of numerical values given events from an event camera. We specifically
investigate the prediction of the 3-DOF angular velocity of a rotating event
camera with an SNN. The difficulty of this problem arises from the prediction
of angular velocities continuously in time directly from irregular,
asynchronous event-based input. Directly utilising the output of event cameras
without any pre-processing ensures that we inherit all the benefits that they
provide over conventional cameras. That is high-temporal resolution,
high-dynamic range and no motion blur. To assess the performance of SNNs on
this task, we introduce a synthetic event camera dataset generated from
real-world panoramic images and show that we can successfully train an SNN to
perform angular velocity regression.

本研究首次提出了使用脉冲神经网络 (SNNs) 对事件相机的数据进行时间回归预测的问题，即预测旋转事件相机的 3 自由度角速度。该研究使用了从真实全景图像生成的合成事件相机数据集，证明了可以成功地训练 SNNs 来执行角速度回归。