In feedback generation for logical errors in programming assignments, large
language model (LLM)-based methods have shown great promise. These methods ask
the LLM to generate feedback given the problem statement and a student's
(buggy) submission. There are several issues with these types of methods.
First, the generated feedback messages are often too direct in revealing the
error in the submission and thus diminish valuable opportunities for the
student to learn. Second, they do not consider the student's learning context,
i.e., their previous submissions, current knowledge, etc. Third, they are not
layered since existing methods use a single, shared prompt for all student
submissions. In this paper, we explore using LLMs to generate a
"feedback-ladder", i.e., multiple levels of feedback for the same
problem-submission pair. We evaluate the quality of the generated
feedback-ladder via a user study with students, educators, and researchers. We
have observed diminishing effectiveness for higher-level feedback and
higher-scoring submissions overall in the study. In practice, our method
enables teachers to select an appropriate level of feedback to show to a
student based on their personal learning context, or in a progressive manner to
go more detailed if a higher-level feedback fails to correct the student's
error.

基于大型语言模型的反馈生成方法在编程作业的逻辑错误中显示出巨大潜力。本研究通过使用多级反馈的方式来解决现有方法存在的问题，并通过用户研究评估了所生成反馈的质量。