Event-based cameras are inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the even data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks that are difficult to train. In this work, we propose a neural network architecture based on simple convolution layers integrated with dynamic temporal encoding reservoirs with low hardware and training costs. The Reservoir-enabled Time Integrated Attention Network (RetinaNet) allows the network to efficiently process asynchronous temporal features, and achieves the highest accuracy of 99.2% for DVS128 Gesture reported to date, and one of the highest accuracy of 67.5% for DVS Lip dataset at a much smaller network size. By leveraging the internal dynamics of memristors, asynchronous temporal feature encoding can be implemented at very low hardware cost without preprocessing or dedicated memory and arithmetic units. The use of simple DNN blocks and backpropagation based training rules further reduces its implementation cost. Code will be publicly available.

本文提出了一种基于简单卷积层的神经网络架构，结合低成本的动态时间编码水库节点，实现了异步时间特征的高效处理，并通过使用内部动态的自旋电阻器，在非常低的硬件成本下实现了异步时间特征编码。RN-Net 在 DVS128 手势数据集上取得了最高的 99.2% 的准确率，并且在较小的网络规模下，在 DVS Lip 数据集上实现了达到 67.5% 的准确率，且代码将公开发布。

RN-Net: 储水池节点启用的神经形态视觉感知网络