Large language models such as ChatGPT-3.5 and GPT-4.0 are ubiquitous and
dominate the current discourse. Their transformative capabilities have led to a
paradigm shift in how we interact with and utilize (text-based) information.
Each day, new possibilities to leverage the capabilities of these models
emerge. This paper presents findings on the performance of different large
language models in a university of applied sciences' undergraduate computer
science degree program. Our primary objective is to assess the effectiveness of
these models within the curriculum by employing them as educational aids. By
prompting the models with lecture material, exercise tasks, and past exams, we
aim to evaluate their proficiency across different computer science domains. We
showcase the strong performance of current large language models while
highlighting limitations and constraints within the context of such a degree
program. We found that ChatGPT-3.5 averaged 79.9% of the total score in 10
tested modules, BingAI achieved 68.4%, and LLaMa, in the 65 billion parameter
variant, 20%. Despite these convincing results, even GPT-4.0 would not pass the
degree program - due to limitations in mathematical calculations.

这篇论文在应用科学大专的计算机科学学士学位课程中研究了不同大型语言模型的表现和效果，发现当前的大型语言模型在不同计算机科学领域表现出色，但由于数学计算方面的限制，即使是 GPT-4.0 也无法通过该课程。

计算机科学学位项目中大型语言模型的表现

Performance of Large Language Models in a Computer Science Degree  Program

ChatGPT is an AI language model developed by OpenAI that can understand and
generate human-like text. It can be used for a variety of use cases such as
language generation, question answering, text summarization, chatbot
development, language translation, sentiment analysis, content creation,
personalization, text completion, and storytelling. While ChatGPT has garnered
significant positive attention, it has also generated a sense of apprehension
and uncertainty in academic circles. There is concern that students may
leverage ChatGPT to complete take-home assignments and exams and obtain
favorable grades without genuinely acquiring knowledge. This paper adopts a
quantitative approach to demonstrate ChatGPT's high degree of unreliability in
answering a diverse range of questions pertaining to topics in undergraduate
computer science. Our analysis shows that students may risk self-sabotage by
blindly depending on ChatGPT to complete assignments and exams. We build upon
this analysis to provide constructive recommendations to both students and
instructors.

本文采用定量方法，展示 ChatGPT 在回答涉及本科计算机科学课程相关主题的不同类型的问题时高度不可靠，揭示学生盲目依赖 ChatGPT 完成作业和考试可能面临自我破坏。同时提出对学生和教师的建设性建议。