Large language models (LLMs) have shown remarkable performance across a wide
range of applications, often outperforming human experts. However, deploying
these parameter-heavy models efficiently for diverse inference use cases
requires carefully designed hardware platforms with ample computing, memory,
and network resources. With LLM deployment scenarios and models evolving at
breakneck speed, the hardware requirements to meet SLOs remains an open
research question. In this work, we present an analytical tool, GenZ, to study
the relationship between LLM inference performance and various platform design
parameters. Our analysis provides insights into configuring platforms for
different LLM workloads and use cases. We quantify the platform requirements to
support SOTA LLMs models like LLaMA and GPT-4 under diverse serving settings.
Furthermore, we project the hardware capabilities needed to enable future LLMs
potentially exceeding hundreds of trillions of parameters. The trends and
insights derived from GenZ can guide AI engineers deploying LLMs as well as
computer architects designing next-generation hardware accelerators and
platforms. Ultimately, this work sheds light on the platform design
considerations for unlocking the full potential of large language models across
a spectrum of applications. The source code is available at
this https URL .

研究使用 GenZ 工具来探讨大型语言模型的推理性能与各种平台设计参数之间的关系，并提供配置不同工作负载和用例的平台要求，在不同场景下支持 SOTA LLMs 模型的分析，也展示了为支持未来可能超过数万亿参数的 LLMs 所需的硬件能力预测。