In recent years, reference-based and supervised summarization evaluation
metrics have been widely explored. However, collecting human-annotated
references and ratings are costly and time-consuming. To avoid these
limitations, we propose a training-free and reference-free summarization
evaluation metric. Our metric consists of a centrality-weighted relevance score
and a self-referenced redundancy score. The relevance score is computed between
the pseudo reference built from the source document and the given summary,
where the pseudo reference content is weighted by the sentence centrality to
provide importance guidance. Besides an $F_1$-based relevance score, we also
design an $F_\beta$-based variant that pays more attention to the recall score.
As for the redundancy score of the summary, we compute a self-masked similarity
score with the summary itself to evaluate the redundant information in the
summary. Finally, we combine the relevance and redundancy scores to produce the
final evaluation score of the given summary. Extensive experiments show that
our methods can significantly outperform existing methods on both
multi-document and single-document summarization evaluation.

该论文提出了一种无需参考文献且无需训练的摘要评估度量方法，包括中心性加权相关度分数和自我参考冗余分数。实验结果表明，该方法在单篇和多篇文档摘要评估方面显著优于现有方法。