The advent of Large Language Models (LLM) has revolutionized the efficiency
and speed with which tasks are completed, marking a significant leap in
productivity through technological innovation. As these chatbots tackle
increasingly complex tasks, the challenge of assessing the quality of their
outputs has become paramount. This paper critically examines the output quality
of two leading LLMs, OpenAI's ChatGPT and Google's Gemini AI, by comparing the
quality of programming code generated in both their free versions. Through the
lens of a real-world example coupled with a systematic dataset, we investigate
the code quality produced by these LLMs. Given their notable proficiency in
code generation, this aspect of chatbot capability presents a particularly
compelling area for analysis. Furthermore, the complexity of programming code
often escalates to levels where its verification becomes a formidable task,
underscoring the importance of our study. This research aims to shed light on
the efficacy and reliability of LLMs in generating high-quality programming
code, an endeavor that has significant implications for the field of software
development and beyond.

通过比较 OpenAI 的 ChatGPT 和 Google 的 Gemini AI 在其免费版本中生成的编程代码的质量，以一个真实世界的例子和系统的数据集为支撑，本研究批判性地检验了这两种领先的大型语言模型的输出质量。鉴于它们在生成代码方面的显著能力，这方面的聊天机器人能力成为一项特别引人注目的分析领域。此外，编程代码的复杂性常常升级到需要验证的难度，强调了我们研究的重要性。本研究旨在揭示大型语言模型在生成高质量编程代码方面的功效和可靠性，这对软件开发领域和其他领域具有重要意义。

大型语言模型的编程技能评估

Evaluation of the Programming Skills of Large Language Models

We systematically evaluated the performance of seven large language models in
generating programming code using various prompt strategies, programming
languages, and task difficulties. GPT-4 substantially outperforms other large
language models, including Gemini Ultra and Claude 2. The coding performance of
GPT-4 varies considerably with different prompt strategies. In most LeetCode
and GeeksforGeeks coding contests evaluated in this study, GPT-4 employing the
optimal prompt strategy outperforms 85 percent of human participants.
Additionally, GPT-4 demonstrates strong capabilities in translating code
between different programming languages and in learning from past errors. The
computational efficiency of the code generated by GPT-4 is comparable to that
of human programmers. These results suggest that GPT-4 has the potential to
serve as a reliable assistant in programming code generation and software
development.

GPT-4 在生成编程代码方面表现优异，优于其他大型语言模型，具备在不同编程语言之间翻译和学习的强大能力，同时具备与人类程序员相当的代码生成效率，表明其在编程代码生成和软件开发中具备可靠的助手潜力。