Recent advancements in large language models (LLMs) boasting billions of
parameters have generated a significant demand for efficient deployment in
inference workloads. The majority of existing approaches rely on temporal
architectures that reuse hardware units for different network layers and
operators. However, these methods often encounter challenges in achieving low
latency due to considerable memory access overhead. This paper investigates the
feasibility and potential of model-specific spatial acceleration for LLM
inference on FPGAs. Our approach involves the specialization of distinct
hardware units for specific operators or layers, facilitating direct
communication between them through a dataflow architecture while minimizing
off-chip memory accesses. We introduce a comprehensive analytical model for
estimating the performance of a spatial LLM accelerator, taking into account
the on-chip compute and memory resources available on an FPGA. Through our
analysis, we can determine the scenarios in which FPGA-based spatial
acceleration can outperform its GPU-based counterpart. To enable more
productive implementations of an LLM model on FPGAs, we further provide a
library of high-level synthesis (HLS) kernels that are composable and reusable.
This library will be made available as open-source. To validate the
effectiveness of both our analytical model and HLS library, we have implemented
BERT and GPT2 on an AMD Alveo U280 FPGA device. Experimental results
demonstrate our approach can achieve up to 16.1x speedup when compared to
previous FPGA-based accelerators for the BERT model. For GPT generative
inference, we attain a 2.2x speedup compared to DFX, an FPGA overlay, in the
prefill stage, while achieving a 1.9x speedup and a 5.7x improvement in energy
efficiency compared to the NVIDIA A100 GPU in the decode stage.

通过对大型语言模型的空间加速进行研究，本文介绍了针对特定运算符或层的硬件单元专门化的方法，并通过数据流架构实现它们之间的直接通信，从而最小化片外内存访问延迟。研究通过在 AMD Alveo U280 FPGA 设备上实现 BERT 和 GPT2 模型，实验结果表明相较于之前的 FPGA 加速器，在 BERT 模型方面可以实现 16.1 倍的加速，在 GPT 生成推理方面，在前置阶段相较于 DFX，一个 FPGA 叠加层，在解码阶段相较于 NVIDIA A100 GPU，分别实现 2.2 倍的加速和 5.7 倍的能效提升。