Canonical automatic summary evaluation metrics, such as ROUGE, focus on
lexical similarity which cannot well capture semantics nor linguistic quality
and require a reference summary which is costly to obtain. Recently, there have
been a growing number of efforts to alleviate either or both of the two
drawbacks. In this paper, we present a proof-of-concept study to a weakly
supervised summary evaluation approach without the presence of reference
summaries. Massive data in existing summarization datasets are transformed for
training by pairing documents with corrupted reference summaries. In
cross-domain tests, our strategy outperforms baselines with promising
improvements, and show a great advantage in gauging linguistic qualities over
all metrics.

本文提出了一种无需参考摘要的弱监督自动摘要评估方法，通过将现有摘要数据集中的大量数据与损坏的参考摘要进行配对来训练，实现了横向领域测试，证明了该方法在语言质量评估方面具有很大优势。