Attention mechanisms are becoming increasingly popular, being used in neural
network models in multiple domains such as natural language processing (NLP)
and vision applications, especially at the edge. However, attention layers are
difficult to map onto existing neuro accelerators since they have a much higher
density of non-linear operations, which lead to inefficient utilization of
today's vector units. This work introduces NOVA, a NoC-based Vector Unit that
can perform non-linear operations within the NoC of the accelerators, and can
be overlaid onto existing neuro accelerators to map attention layers at the
edge. Our results show that the NOVA architecture is up to 37.8x more
power-efficient than state-of-the-art hardware approximators when running
existing attention-based neural networks.

本研究介绍了 NOVA，一种基于 NoC 的向量单元，可以在加速器的 NoC 中执行非线性操作，并能覆盖现有的神经加速器，以在边缘计算中映射注意力层。我们的结果表明，当运行现有的基于注意力的神经网络时，NOVA 体系结构比最先进的硬件逼近器高效节能，可提高功率效率高达 37.8 倍。