Traditional sequence-to-sequence (seq2seq) models and other variations of the
attention-mechanism such as hierarchical attention have been applied to the
text summarization problem. Though there is a hierarchy in the way humans use
language by forming paragraphs from sentences and sentences from words,
hierarchical models have usually not worked that much better than their
traditional seq2seq counterparts. This effect is mainly because either the
hierarchical attention mechanisms are too sparse using hard attention or noisy
using soft attention. In this paper, we propose a method based on extracting
the highlights of a document; a key concept that is conveyed in a few
sentences. In a typical text summarization dataset consisting of documents that
are 800 tokens in length (average), capturing long-term dependencies is very
important, e.g., the last sentence can be grouped with the first sentence of a
document to form a summary. LSTMs (Long Short-Term Memory) proved useful for
machine translation. However, they often fail to capture long-term dependencies
while modeling long sequences. To address these issues, we have adapted Neural
Semantic Encoders (NSE) to text summarization, a class of memory-augmented
neural networks by improving its functionalities and proposed a novel
hierarchical NSE that outperforms similar previous models significantly. The
quality of summarization was improved by augmenting linguistic factors, namely
lemma, and Part-of-Speech (PoS) tags, to each word in the dataset for improved
vocabulary coverage and generalization. The hierarchical NSE model on factored
dataset outperformed the state-of-the-art by nearly 4 ROUGE points. We further
designed and used the first GPU-based self-critical Reinforcement Learning
model.

本文提出了一种基于文档精华提取的文本摘要方法，并通过引入语言因素和改进神经网络如 LSTMs 和 Neural Semantic Encoders，利用自我强化学习模型进一步提高了文本摘要的质量，实现并超越了文本摘要领域的最佳性能，其中基于分层 NSE 模型的 ROUGE 值提升近 4 个百分点。