A notable challenge in Multi-Document Summarization (MDS) is the
extremely-long length of the input. In this paper, we present an
extract-then-abstract Transformer framework to overcome the problem.
Specifically, we leverage pre-trained language models to construct a
hierarchical extractor for salient sentence selection across documents and an
abstractor for rewriting the selected contents as summaries. However, learning
such a framework is challenging since the optimal contents for the abstractor
are generally unknown. Previous works typically create pseudo extraction oracle
to enable the supervised learning for both the extractor and the abstractor.
Nevertheless, we argue that the performance of such methods could be restricted
due to the insufficient information for prediction and inconsistent objectives
between training and testing. To this end, we propose a loss weighting
mechanism that makes the model aware of the unequal importance for the
sentences not in the pseudo extraction oracle, and leverage the fine-tuned
abstractor to generate summary references as auxiliary signals for learning the
extractor. Moreover, we propose a reinforcement learning method that can
efficiently apply to the extractor for harmonizing the optimization between
training and testing. Experiment results show that our framework substantially
outperforms strong baselines with comparable model sizes and achieves the best
results on the Multi-News, Multi-XScience, and WikiCatSum corpora.

本文提出一种通过提取 - 摘要 Transformer 框架解决多文档摘要中输入长度过长问题的方法，其中包括分层提取和摘要重写。通过权重和深度强化学习等机制建立了抽取 - 摘要模型，实现了在大型数据集 Multi-News，Multi-XScience 和 WikiCatSum 上具有最佳效果的模型。