Large Language Models (LLMs) have shown promising performance in summary
evaluation tasks, yet they face challenges such as high computational costs and
the Lost-in-the-Middle problem where important information in the middle of
long documents is often overlooked. To address these issues, this paper
introduces a novel approach, Extract-then-Evaluate, which involves extracting
key sentences from a long source document and then evaluating the summary by
prompting LLMs. The results reveal that the proposed method not only
significantly reduces evaluation costs but also exhibits a higher correlation
with human evaluations. Furthermore, we provide practical recommendations for
optimal document length and sentence extraction methods, contributing to the
development of cost-effective yet more accurate methods for LLM-based text
generation evaluation.

该研究提出了一种新方法，即从长文档中提取关键句子，然后通过提示大型语言模型来评估摘要，以解决计算成本高、长文档中的重要信息往往被忽视的问题，研究结果显示该方法不仅显著降低了评估成本，而且与人工评估的相关性更高，此外，我们还提供了关于最佳文档长度和句子提取方法的实用建议，为基于大型语言模型的文本生成评估的成本效益更高且更准确的方法的发展做出了贡献。