The issue of factual consistency in abstractive summarization has attracted
much attention in recent years, and the evaluation of factual consistency
between summary and document has become an important and urgent task. Most of
the current evaluation metrics are adopted from the question answering (QA).
However, the application of QA-based metrics is extremely time-consuming in
practice, causing the iteration cycle of abstractive summarization research to
be severely prolonged. In this paper, we propose a new method called ClozE to
evaluate factual consistency by cloze model, instantiated based on masked
language model(MLM), with strong interpretability and substantially higher
speed. We demonstrate that ClozE can reduce the evaluation time by nearly
96$\%$ relative to QA-based metrics while retaining their interpretability and
performance through experiments on six human-annotated datasets and a
meta-evaluation benchmark GO FIGURE \citep{gabriel2020go}. We also implement
experiments to further demonstrate more characteristics of ClozE in terms of
performance and speed. In addition, we conduct an experimental analysis of the
limitations of ClozE, which suggests future research directions. The code and
models for ClozE will be released upon the paper acceptance.

本研究提出了一种名为 ClozE 的新方法，采用基于掩码语言模型（MLM）实例化的填空模型来评估抽象化摘要与原始文本之间的事实一致性，并通过六个人工注释数据集和元评估基准 GO FIGURE 的实验表明 ClozE 相对于 QA_metrics 可以将评估时间减少近 96％，同时保持其可解释性和性能。