In this paper, we explore the application of large language models (LLMs) for generating code-tracing questions in introductory programming courses. We designed targeted prompts for GPT4, guiding it to generate code-tracing questions based on code snippets and descriptions. We established a set of human evaluation metrics to assess the quality of questions produced by the model compared to those created by human experts. Our analysis provides insights into the capabilities and potential of LLMs in generating diverse code-tracing questions. Additionally, we present a unique dataset of human and LLM-generated tracing questions, serving as a valuable resource for both the education and NLP research communities. This work contributes to the ongoing dialogue on the potential uses of LLMs in educational settings.

我们探讨了在初级编程课程中应用大型语言模型（LLM）生成代码追踪问题的方法，通过设计指导GPT4生成基于代码片段和描述的代码追踪问题的有针对性提示，并建立了一套人工评价指标，用于评估模型生成的问题与人工专家创建的问题的质量。我们的分析揭示了LLMs在生成多样化代码追踪问题方面的能力和潜力，并提供了一个独特的人工和LLM生成的追踪问题数据集，为教育和自然语言处理研究社区提供了宝贵资源。这项工作为关于LLMs在教育环境中潜在用途的持续对话做出了贡献。

探索大型语言模型在初级编程课程中生成追踪代码问题的潜力