Generative AI and large language models hold great promise in enhancing programming education by automatically generating individualized feedback for students. We investigate the role of generative AI models in providing human tutor-style programming hints to help students resolve errors in their buggy programs. Recent works have benchmarked state-of-the-art models for various feedback generation scenarios; however, their overall quality is still inferior to human tutors and not yet ready for real-world deployment. In this paper, we seek to push the limits of generative AI models toward providing high-quality programming hints and develop a novel technique, GPT4Hints-GPT3.5Val. As a first step, our technique leverages GPT-4 as a ``tutor'' model to generate hints -- it boosts the generative quality by using symbolic information of failing test cases and fixes in prompts. As a next step, our technique leverages GPT-3.5, a weaker model, as a ``student'' model to further validate the hint quality -- it performs an automatic quality validation by simulating the potential utility of providing this feedback. We show the efficacy of our technique via extensive evaluation using three real-world datasets of Python programs covering a variety of concepts ranging from basic algorithms to regular expressions and data analysis using pandas library.

使用生成式AI模型和大规模语言模型来生成编程教育中的个性化反馈，为学生提供编程提示以帮助他们解决程序中的错误。通过使用GPT-4作为“导师”模型和符号信息来提高生成质量，然后通过使用GPT-3.5作为“学生”模型来验证提示质量，我们开发了一种名为GPT4Hints-GPT3.5Val的新技术，通过对三个实际数据集进行广泛评估展示了我们技术的有效性。

自动化人类导师式编程反馈：利用GPT-4导师模型生成提示以及GPT-3.5学生模型验证提示