Designing high-quality educational questions is a challenging and
time-consuming task. In this work, we propose a novel approach that utilizes
prompt-based techniques to generate descriptive and reasoning-based questions.
However, current question-answering (QA) datasets are inadequate for conducting
our experiments on prompt-based question generation (QG) in an educational
setting. Therefore, we curate a new QG dataset called EduProbe for school-level
subjects, by leveraging the rich content of NCERT textbooks. We carefully
annotate this dataset as quadruples of 1) Context: a segment upon which the
question is formed; 2) Long Prompt: a long textual cue for the question (i.e.,
a longer sequence of words or phrases, covering the main theme of the context);
3) Short Prompt: a short textual cue for the question (i.e., a condensed
representation of the key information or focus of the context); 4) Question: a
deep question that aligns with the context and is coherent with the prompts. We
investigate several prompt-based QG methods by fine-tuning pre-trained
transformer-based large language models (LLMs), namely PEGASUS, T5, MBART, and
BART. Moreover, we explore the performance of two general-purpose pre-trained
LLMs such as Text-Davinci-003 and GPT-3.5-Turbo without any further training.
By performing automatic evaluation, we show that T5 (with long prompt)
outperforms all other models, but still falls short of the human baseline.
Under human evaluation criteria, TextDavinci-003 usually shows better results
than other models under various prompt settings. Even in the case of human
evaluation criteria, QG models mostly fall short of the human baseline. Our
code and dataset are available at: this https URL

利用基于提示的技术生成描述性和推理性问题是一个具有挑战性和耗时的任务。本研究提出了一种新方法，利用基于提示的技术生成描述性和推理性问题。我们通过精选的 NCERT 教科书的丰富内容，创建了一个名为 EduProbe 的新的问题生成数据集。我们针对此数据集调查了几种基于提示的问题生成方法，并使用预训练的大型语言模型进行了微调。通过自动评估，我们发现 T5（使用长提示）表现优于其他模型，但仍然低于人类基准。在人类评估标准下，TextDavinci-003 通常在各种提示设置下表现更好。即使在人类评估标准下，问题生成模型也大多低于人类基准。