The rapid evolution of artificial intelligence (AI), specifically large
language models (LLMs), has opened opportunities for various educational
applications. This paper explored the feasibility of utilizing ChatGPT, one of
the most popular LLMs, for automating feedback for Java programming assignments
in an introductory computer science (CS1) class. Specifically, this study
focused on three questions: 1) To what extent do students view LLM-generated
feedback as formative? 2) How do students see the comparative affordances of
feedback prompts that include their code, vs. those that exclude it? 3) What
enhancements do students suggest for improving AI-generated feedback? To
address these questions, we generated automated feedback using the ChatGPT API
for four lab assignments in the CS1 class. The survey results revealed that
students perceived the feedback as aligning well with formative feedback
guidelines established by Shute. Additionally, students showed a clear
preference for feedback generated by including the students' code as part of
the LLM prompt, and our thematic study indicated that the preference was mainly
attributed to the specificity, clarity, and corrective nature of the feedback.
Moreover, this study found that students generally expected specific and
corrective feedback with sufficient code examples, but had diverged opinions on
the tone of the feedback. This study demonstrated that ChatGPT could generate
Java programming assignment feedback that students perceived as formative. It
also offered insights into the specific improvements that would make the
ChatGPT-generated feedback useful for students.

ChatGPT 在自动化给予 Java 编程作业反馈方面的可行性进行了研究，调查结果表明学生们普遍认为 ChatGPT 反馈与 Shute 建立的形成性反馈准则相一致，他们更喜欢包含他们代码的反馈，此研究还提供了改进 ChatGPT 生成反馈的具体见解。