There is a constant need for educators to develop and maintain effective
up-to-date assessments. While there is a growing body of research in computing
education on utilizing large language models (LLMs) in generation and
engagement with coding exercises, the use of LLMs for generating programming
MCQs has not been extensively explored. We analyzed the capability of GPT-4 to
produce multiple-choice questions (MCQs) aligned with specific learning
objectives (LOs) from Python programming classes in higher education.
Specifically, we developed an LLM-powered (GPT-4) system for generation of MCQs
from high-level course context and module-level LOs. We evaluated 651
LLM-generated and 449 human-crafted MCQs aligned to 246 LOs from 6 Python
courses. We found that GPT-4 was capable of producing MCQs with clear language,
a single correct choice, and high-quality distractors. We also observed that
the generated MCQs appeared to be well-aligned with the LOs. Our findings can
be leveraged by educators wishing to take advantage of the state-of-the-art
generative models to support MCQ authoring efforts.

通过分析 GPT-4 在高等教育的 Python 编程课程中生成的多项选择题（MCQs），我们发现 GPT-4 能够产生具有清晰语言、单一正确选项和高质量干扰项的 MCQs，并且这些生成的题目与学习目标高度一致。这些研究结果对于希望充分利用最先进生成模型来支持多项选择题的编写工作的教育工作者具有潜在价值。