OpenAI Codex is a recent large language model from the GPT-3 family for translating code into natural language and vice versa. Recent explorations of Codex have highlighted that given typical introductory programming exercise problem statements as input, the model can generate code solutions well above the level of an average student. In this article, we explore the natural language generation capabilities of Codex in two different phases of the life of a programming exercise; automatically creating programming exercises (including sample solutions and test cases) and explanations of written code, assessing these qualitatively and quantitatively. We find the majority of this automatically generated content both novel and sensible, and in many cases ready to use as is. We further find that influencing the content of the created programming exercises is remarkably easy with minor modifications to the input. Our analysis suggests that there is significant value in massive generative machine learning models as a tool for instructors, although some oversight might be needed to ensure the quality of the generated content before it is delivered to students. We further discuss the implications of OpenAI Codex and similar tools for introductory programming education and highlight future research streams that have the potential to improve the quality of the educational experience for both teachers and students alike.

本文研究了利用大型语言模型进行自然语言生成的能力，并应用于编程课程中常见的两种学习资源的制作。研究发现，在创建编程练习时，只需要输入关键词，即可显著影响编程概念和上下文主题的内容，同时也证明了大多数自动生成的内容是全新且合理的。这些结果表明，使用大型生成机器学习模型作为教学工具是有重要价值的，但在交付给学生之前需要一些监督来确保生成的内容质量。文章还探讨了OpenAI Codex及类似工具对初学者编程教育的影响，并强调了可能改善教学体验的未来研究方向。

使用大型语言模型自动生成编程练习和代码解释