Large Language Models (LLMs) have become extremely potent instruments with
exceptional capacities for comprehending and producing human-like text in a
wide range of applications. However, the increasing size and complexity of LLMs
present significant challenges in both training and deployment, leading to
substantial computational and storage costs as well as heightened energy
consumption. In this paper, we provide a review of recent advancements and
research directions aimed at addressing these challenges and enhancing the
efficiency of LLM-based systems. We begin by discussing algorithm-level
acceleration techniques focused on optimizing LLM inference speed and resource
utilization. We also explore LLM-hardware co-design strategies with a vision to
improve system efficiency by tailoring hardware architectures to LLM
requirements. Further, we delve into LLM-to-accelerator compilation approaches,
which involve customizing hardware accelerators for efficient LLM deployment.
Finally, as a case study to leverage LLMs for assisting circuit design, we
examine LLM-aided design methodologies for an important task: High-Level
Synthesis (HLS) functional verification, by creating a new dataset that
contains a large number of buggy and bug-free codes, which can be essential for
training LLMs to specialize on HLS verification and debugging. For each aspect
mentioned above, we begin with a detailed background study, followed by the
presentation of several novel solutions proposed to overcome specific
challenges. We then outline future research directions to drive further
advancements. Through these efforts, we aim to pave the way for more efficient
and scalable deployment of LLMs across a diverse range of applications.

该研究综述了针对大型语言模型（LLMs）的挑战以及提高系统效率的最新进展和研究方向，包括算法级加速技术、LLM 硬件与软件协同设计策略、LLMs 加速器编译方法以及利用 LLMs 辅助电路设计的方法。通过这些工作，旨在为 LLMs 在各种应用中实现更高效、可扩展的部署铺平道路。