A common way of assessing language learners' mastery of vocabulary is via
multiple-choice cloze (i.e., fill-in-the-blank) questions. But the creation of
test items can be laborious for individual teachers or in large-scale language
programs. In this paper, we evaluate a new method for automatically generating
these types of questions using large language models (LLM). The VocaTT
(vocabulary teaching and training) engine is written in Python and comprises
three basic steps: pre-processing target word lists, generating sentences and
candidate word options using GPT, and finally selecting suitable word options.
To test the efficiency of this system, 60 questions were generated targeting
academic words. The generated items were reviewed by expert reviewers who
judged the well-formedness of the sentences and word options, adding comments
to items judged not well-formed. Results showed a 75% rate of well-formedness
for sentences and 66.85% rate for suitable word options. This is a marked
improvement over the generator used earlier in our research which did not take
advantage of GPT's capabilities. Post-hoc qualitative analysis reveals several
points for improvement in future work including cross-referencing
part-of-speech tagging, better sentence validation, and improving GPT prompts.

使用大型语言模型自动生成填空题，并评估其有效性和适用性，通过生成适当的句子和选择合适的答案，以提高学生的词汇掌握水平。