Transformer networks have emerged as the state-of-the-art approach for
natural language processing tasks and are gaining popularity in other domains
such as computer vision and audio processing. However, the efficient hardware
acceleration of transformer models poses new challenges due to their high
arithmetic intensities, large memory requirements, and complex dataflow
dependencies. In this work, we propose ITA, a novel accelerator architecture
for transformers and related models that targets efficient inference on
embedded systems by exploiting 8-bit quantization and an innovative softmax
implementation that operates exclusively on integer values. By computing
on-the-fly in streaming mode, our softmax implementation minimizes data
movement and energy consumption. ITA achieves competitive energy efficiency
with respect to state-of-the-art transformer accelerators with 16.9 TOPS/W,
while outperforming them in area efficiency with 5.93 TOPS/mm$^2$ in 22 nm
fully-depleted silicon-on-insulator technology at 0.8 V.

该论文提出了 ITA，一种新颖的加速器架构，用于 Transformer 模型和相关模型的高效推理，通过利用 8 位量化和一种仅操作整数值的创新 softmax 实现，在嵌入式系统上实现了低功耗和高效能。ITA 在能效上与最先进的 Transformer 加速器相媲美，达到 16.9 TOPS/W，而在面积效率方面以 22 纳米完全耗尽硅上绝缘体技术满足 0.8 V 下的每平方毫米 5.93 TOPS/mm² 的性能。