Neuromorphic Computing promises orders of magnitude improvement in energy
efficiency compared to traditional von Neumann computing paradigm. The goal is
to develop an adaptive, fault-tolerant, low-footprint, fast, low-energy
intelligent system by learning and emulating brain functionality which can be
realized through innovation in different abstraction layers including material,
device, circuit, architecture and algorithm. As the energy consumption in
complex vision tasks keep increasing exponentially due to larger data set and
resource-constrained edge devices become increasingly ubiquitous, spike-based
neuromorphic computing approaches can be viable alternative to deep
convolutional neural network that is dominating the vision field today. In this
book chapter, we introduce neuromorphic computing, outline a few representative
examples from different layers of the design stack (devices, circuits and
algorithms) and conclude with a few exciting applications and future research
directions that seem promising for computer vision in the near future.

神经形态计算有望比传统的冯・诺依曼计算模式在能效上提升数个数量级。其目标是通过学习和模拟大脑功能来开发自适应、容错、低占用、快速、低能耗的智能系统，可以通过在材料、器件、电路、架构和算法等不同抽象层面上进行创新来实现。随着复杂视觉任务的能耗因大型数据集而呈指数增长，并且资源受限的边缘设备变得越来越普遍，基于脉冲的神经形态计算方法可以成为在当前主导视觉领域的深度卷积神经网络的可行替代方案。在本书章节中，我们介绍了神经形态计算，概述了设计堆栈的几个代表性示例（器件、电路和算法），并总结了一些令人兴奋的应用和未来的研究方向，这些方向在近期的计算机视觉中似乎很有前景。

面向下一代计算机视觉的基于脉冲的神经形态计算

Spike-based Neuromorphic Computing for Next-Generation Computer Vision

Spatio-temporal pattern recognition is a fundamental ability of the brain
which is required for numerous real-world activities. Recent deep learning
approaches have reached outstanding accuracies in such tasks, but their
implementation on conventional embedded solutions is still very computationally
and energy expensive. Tactile sensing in robotic applications is a
representative example where real-time processing and energy efficiency are
required. Following a brain-inspired computing approach, we propose a new
benchmark for spatio-temporal tactile pattern recognition at the edge through
Braille letter reading. We recorded a new Braille letters dataset based on the
capacitive tactile sensors of the iCub robot's fingertip. We then investigated
the importance of spatial and temporal information as well as the impact of
event-based encoding on spike-based computation. Afterward, we trained and
compared feedforward and recurrent Spiking Neural Networks (SNNs) offline using
Backpropagation Through Time (BPTT) with surrogate gradients, then we deployed
them on the Intel Loihi neuromorphic chip for fast and efficient inference. We
compared our approach to standard classifiers, in particular to the Long
Short-Term Memory (LSTM) deployed on the embedded NVIDIA Jetson GPU, in terms
of classification accuracy, power, energy consumption, and delay. Our results
show that the LSTM reaches ~97% of accuracy, outperforming the recurrent SNN by
~17% when using continuous frame-based data instead of event-based inputs.
However, the recurrent SNN on Loihi with event-based inputs is ~500 times more
energy-efficient than the LSTM on Jetson, requiring a total power of only ~30
mW. This work proposes a new benchmark for tactile sensing and highlights the
challenges and opportunities of event-based encoding, neuromorphic hardware,
and spike-based computing for spatio-temporal pattern recognition at the edge.

基于 iCub 机器人指尖电容触觉传感器记录了布莱叶盲文字母数据集，以比较前馈和循环脉冲神经网络在 Intel Loihi 神经形态芯片上进行的快速和有效推理，并通过使用代理梯度的 BPTT 进行离线训练和比较，结果表明循环脉冲神经网络在基于事件的输入上比 LSTM 节能～500 倍，为只有～30mW 的总功率。