Large language models trained on code (LLMCs), such as Codex, hold great promise in enhancing programming education by automatically generating feedback for students. We investigate using LLMCs to generate feedback for fixing syntax errors in Python programs, a key scenario in introductory programming. More concretely, given a student's buggy program, our goal is to generate feedback comprising a fixed program along with a natural language explanation describing the errors/fixes, inspired by how a human tutor would give feedback. While using LLMCs is promising, the critical challenge is to ensure high precision in the generated feedback, which is imperative before deploying such technology in classrooms. The main research question we study is: Can we develop LLMCs-based feedback generation techniques with a tunable precision parameter, giving educators quality control over the feedback that students receive? To this end, we introduce PyFiXV, our technique to generate high-precision feedback powered by Codex. The key idea behind PyFiXV is to use a novel run-time validation mechanism to decide whether the generated feedback is suitable for sharing with the student; notably, this validation mechanism also provides a precision knob to educators. We perform an extensive evaluation using two real-world datasets of Python programs with syntax errors and show the efficacy of PyFiXV in generating high-precision feedback.

本文介绍了使用大型编程语言模型（LLMCs和Codex）来生成高精度、人类可理解的反馈，以帮助初学者在Python编程中解决语法错误等问题，并且提出了一种新的运行时验证机制，这种机制能够为教育者提供一个调节精度的旋钮，确保反馈的准确性。通过两个真实数据集的评估表明，这个技术能够在生成高精度反馈方面取得良好的效果。

利用大语言模型为编程语法错误生成高精度反馈