The deployment of Vision Transformers (ViTs) on hardware platforms, specially
Field-Programmable Gate Arrays (FPGAs), presents many challenges, which are
mainly due to the substantial computational and power requirements of their
non-linear functions, notably layer normalization, softmax, and Gaussian Error
Linear Unit (GELU). These critical functions pose significant obstacles to
efficient hardware implementation due to their complex mathematical operations
and the inherent resource count and architectural limitations of FPGAs.
PEANO-ViT offers a novel approach to streamlining the implementation of the
layer normalization layer by introducing a division-free technique that
simultaneously approximates the division and square root function.
Additionally, PEANO-ViT provides a multi-scale division strategy to eliminate
division operations in the softmax layer, aided by a Pade-based approximation
for the exponential function. Finally, PEANO-ViT introduces a piece-wise linear
approximation for the GELU function, carefully designed to bypass the
computationally intensive operations associated with GELU. In our comprehensive
evaluations, PEANO-ViT exhibits minimal accuracy degradation (<= 0.5% for
DeiT-B) while significantly enhancing power efficiency, achieving improvements
of 1.91x, 1.39x, 8.01x for layer normalization, softmax, and GELU,
respectively. This improvement is achieved through substantial reductions in
DSP, LUT, and register counts for these non-linear operations. Consequently,
PEANO-ViT enables efficient deployment of Vision Transformers on resource- and
power-constrained FPGA platforms.

PEANO-ViT 通过引入无除法技术简化了层归一化层的实现，同时提供了多尺度除法策略以消除 softmax 层的除法操作，并通过分段线性逼近来绕过与 GELU 相关的计算密集型操作，从而在减少 DSP、LUT 和寄存器数量的同时，实现了 0.5% 的准确度降低和显著提高 1.91 倍、1.39 倍和 8.01 倍的层归一化、softmax 和 GELU 的功率效率，从而在资源和功耗受限的 FPGA 平台上实现了视觉 Transformer 的高效部署。