This work proposes a novel method to generate C-Tests; a deviated form of
cloze tests (a gap filling exercise) where only the last part of a word is
turned into a gap. In contrast to previous works that only consider varying the
gap size or gap placement to achieve locally optimal solutions, we propose a
mixed-integer programming (MIP) approach. This allows us to consider gap size
and placement simultaneously, achieving globally optimal solutions, and to
directly integrate state-of-the-art models for gap difficulty prediction into
the optimization problem. A user study with 40 participants across four C-Test
generation strategies (including GPT-4) shows that our approach (MIP)
significantly outperforms two of the baseline strategies (based on gap
placement and GPT-4); and performs on-par with the third (based on gap size).
Our analysis shows that GPT-4 still struggles to fulfill explicit constraints
during generation and that MIP produces C-Tests that correlate best with the
perceived difficulty. We publish our code, model, and collected data consisting
of 32 English C-Tests with 20 gaps each (totaling 3,200 individual gap
responses) under an open source license.

该研究论文提出了一种新的方法来生成 C-Tests，它是一种 cloze 测试的变体，在这里只有单词的最后一部分变成了间隙。与以往只考虑变化间隙大小或间隙位置以获得局部最优解的方法相比，我们提出了一种混合整数规划（MIP）方法。这使我们能够同时考虑间隙大小和位置，实现全局最优解，并直接将最先进的间隙难度预测模型集成到优化问题中。通过与四种 C-Test 生成策略（包括 GPT-4）的 40 名参与者的用户研究，我们发现我们的方法（MIP）明显优于两种基准策略（基于间隙位置和 GPT-4），并且与第三种策略（基于间隙大小）表现相当。我们的分析表明，GPT-4 在生成过程中仍然难以满足显式约束条件，而 MIP 生成的 C-Tests 与所感知的困难程度最相关。我们在开源许可下发布了我们的代码、模型和收集的数据，其中包括 32 个英语 C-Tests，每个测试包含 20 个间隙（总共 3200 个单独的间隙响应）。