As large language models continue to scale in size rapidly, so too does the
computational power required to run them. Event-based networks on neuromorphic
devices offer a potential way to reduce energy consumption for inference
significantly. However, to date, most event-based networks that can run on
neuromorphic hardware, including spiking neural networks (SNNs), have not
achieved task performance even on par with LSTM models for language modeling.
As a result, language modeling on neuromorphic devices has seemed a distant
prospect. In this work, we demonstrate the first-ever implementation of a
language model on a neuromorphic device - specifically the SpiNNaker 2 chip -
based on a recently published event-based architecture called the EGRU.
SpiNNaker 2 is a many-core neuromorphic chip designed for large-scale
asynchronous processing, while the EGRU is architected to leverage such
hardware efficiently while maintaining competitive task performance. This
implementation marks the first time a neuromorphic language model matches
LSTMs, setting the stage for taking task performance to the level of large
language models. We also demonstrate results on a gesture recognition task
based on inputs from a DVS camera. Overall, our results showcase the
feasibility of this neuro-inspired neural network in hardware, highlighting
significant gains versus conventional hardware in energy efficiency for the
common use case of single batch inference.

这项研究实现了首个基于神经形态学设备的语言模型，在 SpiNNaker 2 芯片上基于一个名为 EGRU 的最新事件驱动架构。该实现标志着神经形态学语言模型首次能与 LSTM 相媲美，为将任务性能提升到大型语言模型水平奠定了基础。