Recent studies have integrated large language models (LLMs) into diverse educational contexts, including providing adaptive programming hints, a type of feedback focuses on helping students move forward during problem-solving. However, most existing LLM-based hint systems are limited to one single hint type. To investigate whether and how different levels of hints can support students' problem-solving and learning, we conducted a think-aloud study with 12 novices using the LLM Hint Factory, a system providing four levels of hints from general natural language guidance to concrete code assistance, varying in format and granularity. We discovered that high-level natural language hints alone can be helpless or even misleading, especially when addressing next-step or syntax-related help requests. Adding lower-level hints, like code examples with in-line comments, can better support students. The findings open up future work on customizing help responses from content, format, and granularity levels to accurately identify and meet students' learning needs.

近期的研究将大型语言模型（LLMs）应用于多样的教育环境，包括提供自适应编程提示，一种专注于帮助学生在问题解决中前进的反馈类型。本研究通过使用LLM Hint Factory的思辨法研究，对12名初学者进行了研究，该系统提供四个级别的提示，从一般的自然语言引导到具体的代码辅助，格式和粒度不同。我们发现，仅凭高级自然语言提示可能无济于事，甚至具有误导性，尤其是在处理下一步或与语法相关的帮助请求时。添加低级的提示，如带有行内注释的代码示例，可以更好地支持学生。这些发现为定制内容、格式和粒度级别的帮助响应，以准确识别和满足学生的学习需求，开辟了未来的研究方向。

探索多层次的GPT生成的编程提示如何支持或让初学者失望