Generative Artificial Intelligence (GAI) is taking the world by storm with
its unparalleled content creation ability. Large Language Models (LLMs) are at
the forefront of this movement. However, the significant resource demands of
LLMs often require cloud hosting, which raises issues regarding privacy,
latency, and usage limitations. Although edge intelligence has long been
utilized to solve these challenges by enabling real-time AI computation on
ubiquitous edge resources close to data sources, most research has focused on
traditional AI models and has left a gap in addressing the unique
characteristics of LLM inference, such as considerable model size,
auto-regressive processes, and self-attention mechanisms. In this paper, we
present an edge intelligence optimization problem tailored for LLM inference.
Specifically, with the deployment of the batching technique and model
quantization on resource-limited edge devices, we formulate an inference model
for transformer decoder-based LLMs. Furthermore, our approach aims to maximize
the inference throughput via batch scheduling and joint allocation of
communication and computation resources, while also considering edge resource
constraints and varying user requirements of latency and accuracy. To address
this NP-hard problem, we develop an optimal Depth-First Tree-Searching
algorithm with online tree-Pruning (DFTSP) that operates within a feasible time
complexity. Simulation results indicate that DFTSP surpasses other batching
benchmarks in throughput across diverse user settings and quantization
techniques, and it reduces time complexity by over 45% compared to the
brute-force searching method.

通过部署批处理技术和资源有限的边缘设备上的模型量化，我们在此论文中提出了一种针对基于 Transformer 解码器的大语言模型推断的边缘智能优化问题，旨在通过批处理调度和通信、计算资源的联合分配来最大化推断吞吐量，同时考虑边缘资源约束和不同用户对延迟和准确性的需求。为了解决这个 NP-hard 问题，我们开发了一种在可行时间复杂度内运行的带有在线树剪枝的最优深度优先搜索算法（DFTSP）。模拟结果表明，DFTSP 在各种用户设置和量化技术中超越了其他批处理基准，并且与蛮力搜索方法相比，它的时间复杂度降低了 45% 以上。