We introduce three memory-augmented Recurrent Neural Networks (MARNNs) and explore their capabilities on a series of simple language modeling tasks whose solutions require stack-based mechanisms. We provide the first demonstration of neural networks recognizing the generalized Dyck languages, which express the core of what it means to be a language with hierarchical structure. Our memory-augmented architectures are easy to train in an end-to-end fashion and can learn the Dyck languages over as many as six parenthesis-pairs, in addition to two deterministic palindrome languages and the string-reversal transduction task, by emulating pushdown automata. Our experiments highlight the increased modeling capacity of memory-augmented models over simple RNNs, while inflecting our understanding of the limitations of these models.

本文介绍了三种增强记忆的循环神经网络(MARNNs)，通过一系列简单的语言建模任务来探索其能力，可以识别具有层级结构的一般性Dyck语言，并通过模拟下推自动机来学习带有多达六个圆括号对的Dyck语言，此外还包括两个确定性回文语言和字符串反转转导任务。本文还强调了增强记忆模型相对于简单RNN的建模能力增强程度，同时，对这些模型的局限性进行了一些思考。

记忆扩展的循环神经网络可以学习广义 Dyck 语言