Neural models have become successful at producing abstractive summaries that
are human-readable and fluent. However, these models have two critical
shortcomings: they often don't respect the facts that are either included in
the source article or are known to humans as commonsense knowledge, and they
don't produce coherent summaries when the source article is long. In this work,
we propose a novel architecture that extends Transformer encoder-decoder
architecture in order to improve on these shortcomings. First, we incorporate
entity-level knowledge from the Wikidata knowledge graph into the
encoder-decoder architecture. Injecting structural world knowledge from
Wikidata helps our abstractive summarization model to be more fact-aware.
Second, we utilize the ideas used in Transformer-XL language model in our
proposed encoder-decoder architecture. This helps our model with producing
coherent summaries even when the source article is long. We test our model on
CNN/Daily Mail summarization dataset and show improvements on ROUGE scores over
the baseline Transformer model. We also include model predictions for which our
model accurately conveys the facts, while the baseline Transformer model
doesn't.

本文提出了一种新颖的基于 Transformer 编码器 - 解码器结构的体系结构，通过将实体级知识与来自 Wikidata 的结构全球知识结合，改善了摘要不够准确和连贯的问题，实验证明该模型对长文本摘要具有较好的 ROUGE 分数的提高效果。