Document structure is critical for efficient information consumption.
However, it is challenging to encode it efficiently into the modern Transformer
architecture. In this work, we present HIBRIDS, which injects Hierarchical
Biases foR Incorporating Document Structure into the calculation of attention
scores. We further present a new task, hierarchical question-summary
generation, for summarizing salient content in the source document into a
hierarchy of questions and summaries, where each follow-up question inquires
about the content of its parent question-summary pair. We also annotate a new
dataset with 6,153 question-summary hierarchies labeled on long government
reports. Experiment results show that our model produces better
question-summary hierarchies than comparisons on both hierarchy quality and
content coverage, a finding also echoed by human judges. Additionally, our
model improves the generation of long-form summaries from lengthy government
reports and Wikipedia articles, as measured by ROUGE scores.

本文介绍了 HIBRIDS，一种用于将文档结构注入现代 Transformer 计算注意力得分的方法。同时，提出了一个新的任务，即分层问答 - 摘要生成，并标注了一个新的数据集。实验结果表明，与其他方法相比，HIBRIDS 模型在分层问答 - 摘要的质量和内容覆盖范围方面更优，同时能够提高对长篇政府报告和维基百科文章的摘要生成能力，表现以 ROUGE 指标进行评估。