Traditional sequence-to-sequence (seq2seq) models and other variations of the
attention-mechanism such as hierarchical attention have been applied to the
text summarization problem. Though there is a hierarchy in the way humans use
language by forming paragraphs from sentences and sentences from words,
hierarchical models have usually not worked that much better than their
traditional seq2seq counterparts. This effect is mainly because either the
hierarchical attention mechanisms are too sparse using hard attention or noisy
using soft attention. In this paper, we propose a method based on extracting
the highlights of a document; a key concept that is conveyed in a few
sentences. In a typical text summarization dataset consisting of documents that
are 800 tokens in length (average), capturing long-term dependencies is very
important, e.g., the last sentence can be grouped with the first sentence of a
document to form a summary. LSTMs (Long Short-Term Memory) proved useful for
machine translation. However, they often fail to capture long-term dependencies
while modeling long sequences. To address these issues, we have adapted Neural
Semantic Encoders (NSE) to text summarization, a class of memory-augmented
neural networks by improving its functionalities and proposed a novel
hierarchical NSE that outperforms similar previous models significantly. The
quality of summarization was improved by augmenting linguistic factors, namely
lemma, and Part-of-Speech (PoS) tags, to each word in the dataset for improved
vocabulary coverage and generalization. The hierarchical NSE model on factored
dataset outperformed the state-of-the-art by nearly 4 ROUGE points. We further
designed and used the first GPU-based self-critical Reinforcement Learning
model.

本文提出了一种基于文档精华提取的文本摘要方法，并通过引入语言因素和改进神经网络如 LSTMs 和 Neural Semantic Encoders，利用自我强化学习模型进一步提高了文本摘要的质量，实现并超越了文本摘要领域的最佳性能，其中基于分层 NSE 模型的 ROUGE 值提升近 4 个百分点。

阅读、标记和总结：基于层次神经语义编码器的方法

Read, Highlight and Summarize: A Hierarchical Neural Semantic  Encoder-based Approach

Sentence simplification aims to simplify the content and structure of complex
sentences, and thus make them easier to interpret for human readers, and easier
to process for downstream NLP applications. Recent advances in neural machine
translation have paved the way for novel approaches to the task. In this paper,
we adapt an architecture with augmented memory capacities called Neural
Semantic Encoders (Munkhdalai and Yu, 2017) for sentence simplification. Our
experiments demonstrate the effectiveness of our approach on different
simplification datasets, both in terms of automatic evaluation measures and
human judgments.

本文旨在借助神经机器翻译中的一种新方法 —— 神经语义编码器，对句子简化这一任务进行了研究，实验结果表明，该方法在不同简化数据集上的自动评估措施和人工评估方面都表现出了显著的效果。

记忆增强神经网络的句子简化

Sentence Simplification with Memory-Augmented Neural Networks

We present a memory augmented neural network for natural language
understanding: Neural Semantic Encoders. NSE is equipped with a novel memory
update rule and has a variable sized encoding memory that evolves over time and
maintains the understanding of input sequences through read}, compose and write
operations. NSE can also access multiple and shared memories. In this paper, we
demonstrated the effectiveness and the flexibility of NSE on five different
natural language tasks: natural language inference, question answering,
sentence classification, document sentiment analysis and machine translation
where NSE achieved state-of-the-art performance when evaluated on publically
available benchmarks. For example, our shared-memory model showed an
encouraging result on neural machine translation, improving an attention-based
baseline by approximately 1.0 BLEU.

本文提出了一种称为神经语义编码器（NSE）的增强记忆神经网络，它具有新颖的记忆更新规则和一个随时间变化的可变大小编码内存，并通过读入、组合和写入操作来维护对输入序列的理解，NSE 还可以访问多个共享内存，在自然语言推理，问答，句子分类，文档情感分析和机器翻译等五个不同领域的任务中展示了其有效性和灵活性，在公开可用的基准测试中取得了最先进的性能。