Existing metrics for assessing question generation not only require costly
human reference but also fail to take into account the input context of
generation, rendering the lack of deep understanding of the relevance between
the generated questions and input contexts. As a result, they may wrongly
penalize a legitimate and reasonable candidate question when it (i) involves
complicated reasoning with the context or (ii) can be grounded by multiple
evidences in the context. In this paper, we propose $\textbf{QRelScore}$, a
context-aware $\underline{\textbf{Rel}}$evance evaluation metric for
$\underline{\textbf{Q}}$uestion Generation. Based on off-the-shelf language
models such as BERT and GPT2, QRelScore employs both word-level hierarchical
matching and sentence-level prompt-based generation to cope with the
complicated reasoning and diverse generation from multiple evidences,
respectively. Compared with existing metrics, our experiments demonstrate that
QRelScore is able to achieve a higher correlation with human judgments while
being much more robust to adversarial samples.

提出了一种基于上下文的问题生成的相关性评估度量，称为 QRelScore，它使用字级层次匹配和基于句子级提示的生成，能够应对复杂的推理和多个证据的多样化生成，并且与现有度量相比，具有更高的相关性，并且更具鲁棒性。