In the contemporary landscape of computer architecture, the demand for
efficient parallel programming persists, needing robust optimization
techniques. Traditional optimizing compilers have historically been pivotal in
this endeavor, adapting to the evolving complexities of modern software
systems. The emergence of Large Language Models (LLMs) raises intriguing
questions about the potential for AI-driven approaches to revolutionize code
optimization methodologies.
This paper presents a comparative analysis between two state-of-the-art Large
Language Models, GPT-4.0 and CodeLlama-70B, and traditional optimizing
compilers, assessing their respective abilities and limitations in optimizing
code for maximum efficiency. Additionally, we introduce a benchmark suite of
challenging optimization patterns and an automatic mechanism for evaluating
performance and correctness of the code generated by such tools. We used two
different prompting methodologies to assess the performance of the LLMs --
Chain of Thought (CoT) and Instruction Prompting (IP). We then compared these
results with three traditional optimizing compilers, CETUS, PLUTO and ROSE,
across a range of real-world use cases.
A key finding is that while LLMs have the potential to outperform current
optimizing compilers, they often generate incorrect code on large code sizes,
calling for automated verification methods. Our extensive evaluation across 3
different benchmarks suites shows CodeLlama-70B as the superior optimizer among
the two LLMs, capable of achieving speedups of up to 2.1x. Additionally, CETUS
is the best among the optimizing compilers, achieving a maximum speedup of
1.9x. We also found no significant difference between the two prompting
methods: Chain of Thought (Cot) and Instructing prompting (IP).

该论文通过比较分析两种先进的大型语言模型（GPT-4.0 和 CodeLlama-70B）与传统优化编译器在代码优化方面的能力和局限性，以及引入了一套挑战性的优化模式基准和用于评估生成代码性能和正确性的自动机制，发现虽然大型语言模型有超越当前优化编译器的潜力，但在大型代码上常常生成不正确的代码，需要自动化验证方法，其中 CodeLlama-70B 在两种大型语言模型中表现最佳，最大加速能达到 2.1 倍，而 CETUS 在优化编译器中表现最佳，最大加速能达到 1.9 倍。同时，两种提示方法（思考链和指令提示）没有显著差异。

AI 对您的代码进行优化吗？对比当前大型语言模型与经典优化编译器的研究

Should AI Optimize Your Code? A Comparative Study of Current Large  Language Models Versus Classical Optimizing Compilers

As the use of AI-powered applications widens across multiple domains, so do
increase the computational demands. Primary driver of AI technology are the
deep neural networks (DNNs). When focusing either on cloud-based systems that
serve multiple AI queries from different users each with their own DNN model,
or on mobile robots and smartphones employing pipelines of various models or
parallel DNNs for the concurrent processing of multi-modal data, the next
generation of AI systems will have multi-DNN workloads at their core.
Large-scale deployment of AI services and integration across mobile and
embedded systems require additional breakthroughs in the computer architecture
front, with processors that can maintain high performance as the number of DNNs
increases while meeting the quality-of-service requirements, giving rise to the
topic of multi-DNN accelerator design.

研究了使用 AI 技术的多个领域，阐述了深度神经网络的重要性以及针对多模态数据并发处理的多模型或并行 DNN 的管道，提出了新一代 AI 系统将具有多 DNN 工作负载的核心，需要计算机架构的额外突破，能够在 DNN 数量增加时仍保持高性能，同时满足服务质量要求，因此提出了多 DNN 加速器设计的主题。