Ever since the emergence of large language models (LLMs) and related
applications, such as ChatGPT, its performance and error analysis for
programming tasks have been subject to research. In this work-in-progress
paper, we explore the potential of such LLMs for computing educators and
learners, as we analyze the feedback it generates to a given input containing
program code. In particular, we aim at (1) exploring how an LLM like ChatGPT
responds to students seeking help with their introductory programming tasks,
and (2) identifying feedback types in its responses. To achieve these goals, we
used students' programming sequences from a dataset gathered within a CS1
course as input for ChatGPT along with questions required to elicit feedback
and correct solutions. The results show that ChatGPT performs reasonably well
for some of the introductory programming tasks and student errors, which means
that students can potentially benefit. However, educators should provide
guidance on how to use the provided feedback, as it can contain misleading
information for novices.

探索大型语言模型在计算机教育和学习中的潜力，通过分析其对带有程序代码的输入生成的反馈进行研究，以此为目标来帮助学生解决编程任务并识别不同类型的反馈。结果表明，大型语言模型在一些入门编程任务和学生错误方面表现出了合理的性能，但教育者应提供指导，因为其提供的反馈可能对初学者包含误导性信息。