In this work, we present a compact, modular framework for constructing novel
recurrent neural architectures. Our basic module is a new generic unit, the
Transition Based Recurrent Unit (TBRU). In addition to hidden layer
activations, TBRUs have discrete state dynamics that allow network connections
to be built dynamically as a function of intermediate activations. By
connecting multiple TBRUs, we can extend and combine commonly used
architectures such as sequence-to-sequence, attention mechanisms, and
re-cursive tree-structured models. A TBRU can also serve as both an encoder for
downstream tasks and as a decoder for its own task simultaneously, resulting in
more accurate multi-task learning. We call our approach Dynamic Recurrent
Acyclic Graphical Neural Networks, or DRAGNN. We show that DRAGNN is
significantly more accurate and efficient than seq2seq with attention for
syntactic dependency parsing and yields more accurate multi-task learning for
extractive summarization tasks.

本文提出了一种紧凑，模块化的框架，用于构建新的循环神经网络结构，并介绍了一个基本模块 —— 转换型循环单元 (TBRU)，它不仅具有隐藏层激活函数，而且还能够动态地构建网络连接，多个 TBRU 连接后能延伸和组合常用网络，最终实现更准确的多任务学习和提取性摘要，我们称之为 DRAGNN。实验结果表明 DRAGNN 在句法依赖分析方面的准确率和效率都显著优于带注意力机制的 seq2seq 结构，并为提取性摘要任务提供更准确的多任务学习效果。