The task of automatic text summarization has gained a lot of traction due to
the recent advancements in machine learning techniques. However, evaluating the
quality of a generated summary remains to be an open problem. The literature
has widely adopted Recall-Oriented Understudy for Gisting Evaluation (ROUGE) as
the standard evaluation metric for summarization. However, ROUGE has some
long-established limitations; a major one being its dependence on the
availability of good quality reference summary. In this work, we propose the
metric WIDAR which in addition to utilizing the reference summary uses also the
input document in order to evaluate the quality of the generated summary. The
proposed metric is versatile, since it is designed to adapt the evaluation
score according to the quality of the reference summary. The proposed metric
correlates better than ROUGE by 26%, 76%, 82%, and 15%, respectively, in
coherence, consistency, fluency, and relevance on human judgement scores
provided in the SummEval dataset. The proposed metric is able to obtain
comparable results with other state-of-the-art metrics while requiring a
relatively short computational time.

提出了一种名为 WIDAR 的度量标准，该度量标准使用参考摘要和输入文档评估生成摘要的质量，并且预计的度量标准与其他最新技术的度量标准的结果相当，同时需要相对较短的计算时间。