This study evaluates the efficiency of code generation by Large Language Models (LLMs) and measures their performance against human-crafted solutions using a dataset from Leetcode. We compare 18 LLMs, considering factors such as model temperature and success rate, and their impact on code performance. This research introduces a novel method for measuring and comparing the speed of LLM-generated code, revealing that LLMs produce code with comparable performance, irrespective of the adopted LLM. We also find that LLMs are capable of generating code that is, on average, more efficient than the code written by humans. The paper further discusses the use of Leetcode as a benchmarking dataset, the limitations imposed by potential data contamination, and the platform's measurement reliability. We believe that our findings contribute to a better understanding of LLM capabilities in code generation and set the stage for future optimizations in the field.

本研究评估了大型语言模型（LLMs）生成代码的效率，并将其与人类编写的解决方案进行比较，填补了当前研究中的数据评估空白。提出了一种新方法来测量和比较LLM生成代码的速度，发现LLM生成的代码在性能上与人类代码相当，且平均更为高效。研究结果为理解LLM在代码生成中的能力提供了重要见解，并为未来优化奠定基础。

大型语言模型生成代码在Leetcode上的性能研究