Addressing the challenge of generating personalized feedback for programming
assignments is demanding due to several factors, like the complexity of code
syntax or different ways to correctly solve a task. In this experimental study,
we automated the process of feedback generation by employing OpenAI's GPT-3.5
model to generate personalized hints for students solving programming
assignments on an automated assessment platform. Students rated the usefulness
of GPT-generated hints positively. The experimental group (with GPT hints
enabled) relied less on the platform's regular feedback but performed better in
terms of percentage of successful submissions across consecutive attempts for
tasks, where GPT hints were enabled. For tasks where the GPT feedback was made
unavailable, the experimental group needed significantly less time to solve
assignments. Furthermore, when GPT hints were unavailable, students in the
experimental condition were initially less likely to solve the assignment
correctly. This suggests potential over-reliance on GPT-generated feedback.
However, students in the experimental condition were able to correct reasonably
rapidly, reaching the same percentage correct after seven submission attempts.
The availability of GPT hints did not significantly impact students' affective
state.

本实验研究使用 OpenAI 的 GPT-3.5 模型在一个自动化的评估平台上生成学生编程作业的个性化提示，实验小组依赖平台反馈较少但在启用 GPT 提示时表现得更好。