Despite rapid advancements in large language models (LLMs), QG remains a
challenging problem due to its complicated process, open-ended nature, and the
diverse settings in which question generation occurs. A common approach to
address these challenges involves fine-tuning smaller, custom models using
datasets containing background context, question, and answer. However,
obtaining suitable domain-specific datasets with appropriate context is often
more difficult than acquiring question-answer pairs. In this paper, we
investigate training QG models using synthetic contexts generated by LLMs from
readily available question-answer pairs. We conduct a comprehensive study to
answer critical research questions related to the performance of models trained
on synthetic contexts and their potential impact on QG research and
applications. Our empirical results reveal: 1) contexts are essential for QG
tasks, even if they are synthetic; 2) fine-tuning smaller language models has
the capability of achieving better performances as compared to prompting larger
language models; and 3) synthetic context and real context could achieve
comparable performances. These findings highlight the effectiveness of
synthetic contexts in QG and paves the way for future advancements in the
field.

通过使用大规模语言模型生成的合成上下文进行训练，研究表明，虚拟上下文在问句生成中具有类似实际上下文的效果，为未来该领域的进一步发展铺平了道路。

问题生成的合成上下文生成

Synthetic Context Generation for Question Generation

Conventional automatic evaluation metrics, such as BLEU and ROUGE, developed
for natural language generation (NLG) tasks, are based on measuring the n-gram
overlap between the generated and reference text. These simple metrics may be
insufficient for more complex tasks, such as question generation (QG), which
requires generating questions that are answerable by the reference answers.
Developing a more sophisticated automatic evaluation metric, thus, remains as
an urgent problem in QG research. This work proposes a Prompting-based Metric
on ANswerability (PMAN), a novel automatic evaluation metric to assess whether
the generated questions are answerable by the reference answers for the QG
tasks. Extensive experiments demonstrate that its evaluation results are
reliable and align with human evaluations. We further apply our metric to
evaluate the performance of QG models, which shows our metric complements
conventional metrics. Our implementation of a ChatGPT-based QG model achieves
state-of-the-art (SOTA) performance in generating answerable questions.

本研究提出了一种基于提示的回答可行性度量（PMAN），它是一种新颖的自动评估度量，用于评估生成的问题是否可由参考答案回答，以解决复杂任务如问题生成所面临的评估问题。通过广泛的实验，证明了其评估结果可靠且与人类评估一致。我们还将该度量应用于评估问题生成模型的性能，结果表明该度量与传统度量相辅相成。我们基于 ChatGPT 的问题生成模型达到了最先进的性能。