This research examines the effectiveness of OpenAI's GPT models as
independent evaluators of text summaries generated by six transformer-based
models from Hugging Face: DistilBART, BERT, ProphetNet, T5, BART, and PEGASUS.
We evaluated these summaries based on essential properties of high-quality
summary - conciseness, relevance, coherence, and readability - using
traditional metrics such as ROUGE and Latent Semantic Analysis (LSA). Uniquely,
we also employed GPT not as a summarizer but as an evaluator, allowing it to
independently assess summary quality without predefined metrics. Our analysis
revealed significant correlations between GPT evaluations and traditional
metrics, particularly in assessing relevance and coherence. The results
demonstrate GPT's potential as a robust tool for evaluating text summaries,
offering insights that complement established metrics and providing a basis for
comparative analysis of transformer-based models in natural language processing
tasks.

本研究通过使用传统的 ROUGE 和潜在语义分析（LSA）等指标，将 OpenAI 的 GPT 模型作为独立的评估者，评估了 Hugging Face 的六个基于 Transformer 的模型生成的文本摘要的效果。独特地，我们将 GPT 用作评估器而非摘要生成器，使其能够独立评估摘要的质量，而无需预定义的指标。我们的分析显示 GPT 评估与传统指标之间存在显著相关性，尤其是在评估相关性和连贯性方面。结果展示了 GPT 作为评估文本摘要的强大工具的潜力，提供了对已建立指标的补充洞察，并为自然语言处理任务中基于 Transformer 的模型的比较分析提供了依据。