The field of efficient Large Language Model (LLM) inference is rapidly
evolving, presenting a unique blend of opportunities and challenges. Although
the field has expanded and is vibrant, there hasn't been a concise framework
that analyzes the various methods of LLM Inference to provide a clear
understanding of this domain. Our survey stands out from traditional literature
reviews by not only summarizing the current state of research but also by
introducing a framework based on roofline model for systematic analysis of LLM
inference techniques. This framework enables identifying the bottlenecks in LLM
deployments and provides a deeper understanding of the practical aspects on
real devices, thereby informing more effective strategies for deploying LLM.
Furthermore, we systematically collate the latest advancements in efficient LLM
inference, covering crucial areas such as weight optimization (e.g., Knowledge
Distillation and Quantization), decoding algorithm improvements (e.g., Early
Exit and Mixture-of-Expert), and both hardware and system-level enhancements.
Distinguished by the integration of roofline model analysis, our survey
provides a comprehensive and nuanced exploration of efficient LLM inference
challenges and solutions. This distinctive approach not only showcases the
current research landscape but also delivers valuable insights for practical
implementation, positioning our work as an indispensable resource for
researchers new to the field as well as for those seeking to deepen their
understanding of efficient LLM deployment. The tool LLM-Viewer is open-sourced.

大型语言模型 (LLM) 推理的高效领域正迅速发展，我们的研究通过基于屋顶线模型的框架，系统分析 LLM 推理技术，鉴别 LLM 部署瓶颈，深入理解实际设备的实际方面，并综合整理权重优化、解码算法改进和硬件系统增强等关键领域的最新进展，提供了全面而细致的高效 LLM 推理挑战和解决方案探索。