Large language models (LLMs) excel in zero-shot abstractive summarization
tasks, delivering fluent and pertinent summaries. Recent advancements have
extended their capabilities to handle long-input contexts, surpassing token
limits of 32k or more. However, in the realm of multi-document question
answering, language models exhibit uneven utilization of their input context.
They tend to favor the initial and final segments, resulting in a U-shaped
performance pattern concerning where the answer is located within the input.
This bias raises concerns, particularly in summarization tasks where crucial
content may be dispersed throughout the source document(s). This paper presents
a comprehensive investigation encompassing 10 datasets, 4 LLMs, and 5
evaluation metrics to analyze how these models leverage their input for
abstractive summarization. Our findings reveal a pronounced bias towards the
introductory content (and to a lesser extent, the final content), posing
challenges for LLM performance across a range of diverse summarization
benchmarks.

对于大型语言模型，在抽象摘要任务中表现出色，但在多文档问答中存在输入上下文偏差，导致摘要内容分散，影响性能。本文通过实证研究揭示了这种偏差对于大型语言模型在不同摘要评估上的挑战。