Large Language Models (LLMs) represent a leap in artificial intelligence,
excelling in tasks using human language(s). Although the main focus of
general-purpose LLMs is not code generation, they have shown promising results
in the domain. However, the usefulness of LLMs in an academic software
engineering project has not been fully explored yet. In this study, we explore
the usefulness of LLMs for 214 students working in teams consisting of up to
six members. Notably, in the academic course through which this study is
conducted, students were encouraged to integrate LLMs into their development
tool-chain, in contrast to most other academic courses that explicitly prohibit
the use of LLMs.
In this paper, we analyze the AI-generated code, prompts used for code
generation, and the human intervention levels to integrate the code into the
code base. We also conduct a perception study to gain insights into the
perceived usefulness, influencing factors, and future outlook of LLM from a
computer science student's perspective. Our findings suggest that LLMs can play
a crucial role in the early stages of software development, especially in
generating foundational code structures, and helping with syntax and error
debugging. These insights provide us with a framework on how to effectively
utilize LLMs as a tool to enhance the productivity of software engineering
students, and highlight the necessity of shifting the educational focus toward
preparing students for successful human-AI collaboration.

论文研究了大型语言模型（LLMs）在学术软件工程项目中的实用性，包括 AI 生成的代码、代码生成的提示以及将代码集成到代码库中的人工干预水平。研究结果表明，LLMs 在软件开发的早期阶段，特别是在生成基础代码结构和语法、错误调试方面，可以发挥重要作用。这些发现为我们提供了一个有效利用 LLMs 提高软件工程学生的生产力的框架，并强调了将教育重点转向为学生成功进行人工智能协作的必要性。