In comparison to single-document summarization, abstractive Multi-Document
Summarization (MDS) brings challenges on the representation and coverage of its
lengthy and linked sources. This study develops a Parallel Hierarchical
Transformer (PHT) with attention alignment for MDS. By incorporating word- and
paragraph-level multi-head attentions, the hierarchical architecture of PHT
allows better processing of dependencies at both token and document levels. To
guide the decoding towards a better coverage of the source documents, the
attention-alignment mechanism is then introduced to calibrate beam search with
predicted optimal attention distributions. Based on the WikiSum data, a
comprehensive evaluation is conducted to test improvements on MDS by the
proposed architecture. By better handling the inner- and cross-document
information, results in both ROUGE and human evaluation suggest that our
hierarchical model generates summaries of higher quality relative to other
Transformer-based baselines at relatively low computational cost.

本研究提出了一种带有 Attention Alignment 机制的 Parallel Hierarchical Transformer（PHT）用于 Multi-Document Summarization（MDS）。 通过将单词级和段落级的 multi-head attention 结构综合起来，PHT 的分层体系结构可以更好地处理令牌和文档级别的依赖关系，从而更好地处理内部和跨文档信息。 基于 WikiSum 数据，对所提出的体系结构进行了全面评估，结果表明，相对于其他基于 Transformer 的基线，我们的分层模型可以以相对较低的计算成本，通过更好的处理 MDS 来生成更高质量的摘要。