Question generation (QG) is the task of generating a valid and fluent
question based on a given context and the target answer. According to various
purposes, even given the same context, instructors can ask questions about
different concepts, and even the same concept can be written in different ways.
However, the evaluation for QG usually depends on single reference-based
similarity metrics, such as n-gram-based metric or learned metric, which is not
sufficient to fully evaluate the potential of QG methods. To this end, we
propose to paraphrase the reference question for a more robust QG evaluation.
Using large language models such as GPT-3, we created semantically and
syntactically diverse questions, then adopt the simple aggregation of the
popular evaluation metrics as the final scores. Through our experiments, we
found that using multiple (pseudo) references is more effective for QG
evaluation while showing a higher correlation with human evaluations than
evaluation with a single reference.

提出使用 GPT-3 等大型语言模型来进行问题生成并采用多个（伪）参考答案进行评估，以更全面地评估 QG 技术潜力的方法。实验结果表明，使用多个参考答案进行 QG 评估比使用单个参考答案更为有效，并且更能与人类评估相符合。

评估问题生成需要更多参考文献

Evaluation of Question Generation Needs More References

Neural text generation, including neural machine translation, image
captioning, and summarization, has been quite successful recently. However,
during training time, typically only one reference is considered for each
example, even though there are often multiple references available, e.g., 4
references in NIST MT evaluations, and 5 references in image captioning data.
We first investigate several different ways of utilizing multiple human
references during training. But more importantly, we then propose an algorithm
to generate exponentially many pseudo-references by first compressing existing
human references into lattices and then traversing them to generate new
pseudo-references. These approaches lead to substantial improvements over
strong baselines in both machine translation (+1.5 BLEU) and image captioning
(+3.1 BLEU / +11.7 CIDEr).

利用多个参考文献训练神经文本生成模型的方法对机器翻译和图像字幕生成任务产生了显著的改进，其中一种算法先将现有的参考文献压缩成格点再遍历来生成伪参考文献。